CloudGuard secondary manager in Azure cannot connect to AWS controller

[AK2]

Hi,

R81.20 CloudGuard environment.

Primary Manager is in AWS and manages CloudGuard autoscaling group and Azure VMSS correctly.

I deployed a secondary CloudGuard manager in Azure, added it as a secondary, and did a DB sync.

The secondary has connectivity to all the gateways (AWS and Azure), the primary SMS, and the internet (tested, verified, including AWS URLs mentioned in CME admin guide)

CME config replicated successfully to the secondary.

I stop and disable the CME service on the Primary (Active) SMS per the CME guide

I start and enable the CME service on the Secondary (Standby) SMS per the CME guide

I make the secondary active, which succeeds.

To verify I run "service cme test" on the (now active) secondary SMS.

I get the error below:

testing gwlb-controller... . Time difference is 0:00:00.280533 failed to generate token ERROR: Controller gwlb-controller failed ERROR details: __init__() missing 1 required positional argument: 'body' Testing management configuration... Testing management connectivity... ********** Tests finished **********

I enabled debugging in CME and looked at the logs - nothing obvious.

I have raised a TAC case but no response yet.

NB I don't think the time difference is the cause, when I run a (successful) cme service test on the primary SMS there is also a small time difference but all tests pass.

The Azure SMS uses a different public IP compared to the AWS SMS, I am wondering if the IAM identity associated with how CME logs on to AWS is being refused because the IP is different? It was created using vanilla cloudformation templates and I can't see any evidence of an IP block. As far as I can tell the credentials used by CME on both SMS are the same.

Note - I tried to get round this by creating a new secondary SMS and doing migrate_server, however this reproduced the error above (and caused other errors).

Any help/suggestions for troubleshooting/progressing the issue would be appreciated.

Cheers.

Andrew

[AK2]

From the CME guide:

• Make sure that the clock on the Security Management Server is set correctly.

  The best way to set the clock is with the NTP.

  You need a synchronized clock to make API calls into a cloud environment.

Note that the manager giving the trouble is not ntp synced.

[AK2]

The secondary manager is ntp synced. Error persists.

[AK2]

I removed the secondary manager in Azure. I added a secondary manager in AWS to see if this would be different. The AWS secondary manager is ntp synced.

"service cme test" gives a different error after promoting to primary and following the procedure in the CME guide for HA.

Time difference is 0:00:00.997256
ERROR: Controller gwlb-controller failed
ERROR details: no role in meta-data

[Roman_Kats]

Hello @AK2 
It looks like your CME configuration -> Controller (Account) set to use IAM role.
And your secondary management EC2 instance doesn't have IAM role attached to it (see attached image)

If you want to have secondary management in Azure, an access and secret keys should be used instead of IAM role
Thanks,
Roman

[AK2]

Hi Roman,

Thanks very much for this information, this is the first pointer I have received.

The first SMS was deployed using the CloudFormation templates.

The second SMS was build using standard CloudGuard image and then synced by adding it as a secondary manager.

Most of the CME config synced automatically during the DB sync, ie, the Azure controller synced and works correctly.

Are you saying that the CME config for AWS copied during database sync "won't work" if the AWS controller credentials are IAM based? (I assume, deployed as part of Cloud Formation).

I will research this later, would be helpful if you could point to documentation on how I could:

a) give the secondary management "access and secret keys" so that it can connect to the AWS ASG

b) if required, migrate the primary from "IAM" to "access and secret keys"

Cheers,

Andrew

[Roman_Kats · Accepted solution]

Are you saying that the CME config for AWS copied during database sync "won't work" if the AWS controller credentials are IAM based? (I assume, deployed as part of Cloud Formation).

Answer:
The IAM role designed so that applications can securely make API requests from your instances, without requiring you to manage the security credentials that the applications use. But the instance can be AWS only.
When CME runs on secondary management in Azure it can't use the IAM role to authorize and make API request to AWS account (as IAM is AWS Identity).
Therefore Access and secret keys should be used.

As for error that you get for secondary management in AWS, it doesn't have IAM role attached to it:
Time difference is 0:00:00.997256
ERROR: Controller gwlb-controller failed
ERROR details: no role in meta-data

For more information about AWS authentication methods refer to:
Refer to sk130372 > 3. Creating an AWS IAM User and IAM Role section.

For configuring AWS controller in CME to use access and secret keys refer to CME admin guide:
https://sc1.checkpoint.com/documents/IaaS/WebAdminGuides/EN/CP_CME/Content/Topics-CME/CME_Structure_... 

Example of configuring   CME AWS controller with access and secret key via autoprov_cfg command line utility:
autoprov_cfg add controller AWS -cn <NAME> -r eu-west-1,eu-central-1 -ak <ACCESS-KEY> -sk <SECRET-KEY>

Thanks,
Roman

[AK2]

Thanks Roman. Confirmed that when I add the IAM role to the EC2 instance which is the secondary SMS, "cme service test" functions without error. Now I just need to figure out how to change it to be credentials based so I can get the Azure SMS functional. Will look at sk130372 now. Cheers, Andrew