When running $FWDIR/scripts/azure_ha_test.py on both clsuter members of a newly provisioned AZR Cloudguard HA cluster, when the script tests authorization on routing tables, its reporting the client does not have the \'Microsoft.Network/virtualNetworks/read\' permission on the VNET resource.
This is correct. The cluster managed identities have not been granted access on the VNETs that are peered with the connectivity VNET where the frontend and backend subnets reside.
sk175023 indicates that if the old cluster solution is used, the Contributor role for the cluster managed identities needs to be granted on all VNETS peers to the connectivity VNET
I'm deploying this solution using the master branch of the following repo.
https://github.com/CheckPointSW/CloudGuardIaaS/tree/master/terraform/azure/high-availability-existin...
The output of cat /etc/cloud-version | grep template_name is 'template_name: ha_terraform'
[Expert@cg-cluster-1-demo21:0]# cat /etc/cloud-version | grep template_name
template_name: ha_terraform
A lot of VNETs will be peered with the connectivity (cluster) VNET when this solution is deployed in to production.
If possible, I want to avoid granted Contributor access for the cluster managed identities on each peered VNET.
---------------------------------------------------------------------------------------------------------
sk175023
The tester fails with the "Attempting to write - [Forbidden] Error: HTTP/1.1 403 Forbidden" error
Cause: The cluster members do not have Contributor permissions for their VNET or/and their NICs or/and the Cluster IP address or/and the Network Security Group.
How to resolve:
Navigate to the cluster's VNET resource group or/and the cluster's resource group (in case of Network Security Group from a different resource Group - navigate to the NSG resource).
Assign the following Azure's permissions for each cluster’s managed identity created by the deployment
Contributor to the cluster managed identity named <CLUSTER_NAME1>
Contributor to the cluster managed identity named <CLUSTER_NAME2>
Wait few minutes for the changes to take place (up to one hour).
Note: If you use the old cluster solution, you need to perform the steps above for all the VNET's peered to the cluster VNET.
To check if you use the old cluster solution, run cat /etc/cloud-version | grep template_name and the output should be cluster.
---------------------------------------------------------------------------------------------------------
Output of $FWDIR/scripts/azure_ha_test.py
[Expert@cg-cluster-1-demo22:0]# /opt/CPsuite-R81.20/fw1/scripts/azure_ha_test.py
Setting api versions for "ha_terraform" solution
ARM versions are: {
"resources": "?api-version=2019-07-01"
}
Testing if DNS is configured...
- Primary DNS server is: 10.188.243.12
Testing if DNS is working...
- DNS resolving test was successful
Testing connectivity to login.windows.net:443...
Testing ClusterXL parameters...
Testing cluster interface configuration...
Testing credentials...
Getting information about the environment...
Getting information about the VM cg-cluster-1-demo22...
Id : /subscriptions/<subscription id>/resourceGroups/cloudguard-ha-demo2/providers/Microsoft.Network/networkInterfaces/cg-cluster-1-demo22-eth0
Subscription : <subscription id>
Resource group: cloudguard-ha-demo2
Type : Microsoft.Network/networkInterfaces
Name : cg-cluster-1-demo22-eth0
Attempting to read - [OK]
Attempting to write - [OK]
Id : /subscriptions/<subscription id>/resourceGroups/cloudguard-ha-demo2/providers/Microsoft.Network/networkInterfaces/cg-cluster-1-demo22-eth1
Subscription : <subscription id>
Resource group: cloudguard-ha-demo2
Type : Microsoft.Network/networkInterfaces
Name : cg-cluster-1-demo22-eth1
Attempting to read - [OK]
Attempting to write - [OK]
Getting information about the VM cg-cluster-1-demo21...
Id : /subscriptions/<subscription id>/resourceGroups/cloudguard-ha-demo2/providers/Microsoft.Network/networkInterfaces/cg-cluster-1-demo21-eth0
Subscription : <subscription id>
Resource group: cloudguard-ha-demo2
Type : Microsoft.Network/networkInterfaces
Name : cg-cluster-1-demo21-eth0
Attempting to read - [OK]
Attempting to write - [OK]
Id : /subscriptions/<subscription id>/resourceGroups/cloudguard-ha-demo2/providers/Microsoft.Network/networkInterfaces/cg-cluster-1-demo21-eth1
Subscription : <subscription id>
Resource group: cloudguard-ha-demo2
Type : Microsoft.Network/networkInterfaces
Name : cg-cluster-1-demo21-eth1
Attempting to read - [OK]
Attempting to write - [OK]
Testing authorization on routing tables...
Failed to retrieve peered network /subscriptions/<subscription id>/resourceGroups/demo-rg/providers/Microsoft.Network/virtualNetworks/demo-dev-vnet-01
Traceback (most recent call last):
File "/opt/CPsuite-R81.20/fw1/scripts/azure_ha_test.py", line 166, in get_route_table_ids_for_peering
vnet = azure.arm('GET', vnet_id)[1]
File "/opt/CPsuite-R81.20/fw1/scripts/rest.py", line 624, in arm
max_time=self.max_time)
File "/opt/CPsuite-R81.20/fw1/scripts/rest.py", line 200, in request
response)
rest.RequestException: HTTP/1.1 403 Forbidden
b'{"error":{"code":"AuthorizationFailed","message":"The client \'fc940c4c-4a2f-4428-99d5-99a1b4d96305\' with object id \'fc940c4c-4a2f-4428-99d5-99a1b4d96305\' does not have authorization to perform action \'Microsoft.Network/virtualNetworks/read\' over scope \'/subscriptions/<subscription id>/resourceGroups/demo-rg/providers/Microsoft.Network/virtualNetworks/demo-dev-vnet-01\' or the scope is invalid. If access was recently granted, please refresh your credentials."}}'
Failed to retrieve peered network /subscriptions/<subscription id>/resourceGroups/demo-rg/providers/Microsoft.Network/virtualNetworks/demo-dev-vnet-02
Traceback (most recent call last):
File "/opt/CPsuite-R81.20/fw1/scripts/azure_ha_test.py", line 166, in get_route_table_ids_for_peering
vnet = azure.arm('GET', vnet_id)[1]
File "/opt/CPsuite-R81.20/fw1/scripts/rest.py", line 624, in arm
max_time=self.max_time)
File "/opt/CPsuite-R81.20/fw1/scripts/rest.py", line 200, in request
response)
rest.RequestException: HTTP/1.1 403 Forbidden
b'{"error":{"code":"AuthorizationFailed","message":"The client \'fc940c4c-4a2f-4428-99d5-99a1b4d96305\' with object id \'fc940c4c-4a2f-4428-99d5-99a1b4d96305\' does not have authorization to perform action \'Microsoft.Network/virtualNetworks/read\' over scope \'/subscriptions/<subscription id>/resourceGroups/demo-rg/providers/Microsoft.Network/virtualNetworks/demo-dev-vnet-02\' or the scope is invalid. If access was recently granted, please refresh your credentials."}}'
Id : /subscriptions/<subscription id>/resourceGroups/demo-rg/providers/Microsoft.Network/routeTables/azne-external-routetable
Subscription : <subscription id>
Resource group: demo-rg
Type : Microsoft.Network/routeTables
Name : azne-external-routetable
Attempting to read - [Forbidden]
Error:
HTTP/1.1 403 Forbidden
b'{"error":{"code":"AuthorizationFailed","message":"The client \'fc940c4c-4a2f-4428-99d5-99a1b4d96305\' with object id \'fc940c4c-4a2f-4428-99d5-99a1b4d96305\' does not have authorization to perform action \'Microsoft.Network/routeTables/read\' over scope \'/subscriptions/<subscription id>/resourceGroups/demo-rg/providers/Microsoft.Network/routeTables/azne-external-routetable\' or the scope is invalid. If access was recently granted, please refresh your credentials."}}'
