CloudGuard IaaS HA BGP Setup with Azure Route Server

[Ave_Joe]

Does anyone have a working BGP configuration for peering a CloudGuard IaaS HA setup with an Azure Route Server that they could share?

I'm deploying Azure VMware Solution (AVS) in Azure, which requires BGP peering between my CloudGuard IaaS HA setup and an Azure Route Server in the AVS connectivity hub. I believe the solution involves two main components: (1) establishing BGP peering with the Azure Route Server and (2) configuring the route map correctly.

I've reviewed the Advanced Routing Guide and the Deep Dive article "CloudGuard + Azure Route Server = Easier Security Administration", but I'm still struggling to get the configuration working as needed. I currently have a TAC case open, hoping they might provide additional documentation that lays out a complete solution in one place, but I haven’t found anything comprehensive so far.

Any guidance, or an example configuration, would be greatly appreciated. Thank you!

[Jeff_Engel]

Talked to the one and only @Gustavo_Coronel  and here is what he had to say...we are also more than happy to set up a direct call to discuss further in detail.

I can’t be 100% sure unless I see a diagram.

BUT… If it is a standard AVS connection using Express Route (and Global Reach to connect the on-prem networks), all you need to do is advertise the CloudGuard ILB IP as the default route into the ARS.

The screenshot below is from the deep dive CloudGuard + Azure Route Server = Easier Security Administration video and it includes the pertinent (Advertise ILB as next hop for Default Gateway) route map.

Replace the IPs on the screenshot with the correct ones and make sure the BGP peering with the ARS is established (AND that the ARS has "Branch to Branch” enabled).

It is on the video.

If you have a different architecture, then it may be different, but can’t say without looking at a diagram.

[Ave_Joe]

Hi Jeff:

Here is the a diagram that represents what is trying to be done with BGP in my setup.  At this time the only path in and out of Azure is through the Cloud Guard HA gateway currently deployed.  An Express Route to the Data Center will be added later. 

Take a look.  I can add any information that may be missing.  If a call makes more sense I am up for it.

Let me know.

 

[Jeff_Engel · Accepted solution]

Where is the future ExpressRoute coming from OnPrem terminating?

Hopefully, it will be terminating on the 10.111.0.0/24 VNET. Otherwise, you might be painting yourself into a corner.

If that is the plan, then the peering between the CloudGuard VNET and the 10.111.0.0/24 VNET must be configured so that the CloudGuard VNET can use the Remote Gateways in 10.111.0.0/24.

If that is indeed the plan, all you need to do is:

Peer the CloudGuard HA members with the ARS in 10.111.0.0/24 using the eth0 interface for each member  (DO NOT USE THE VIP and DO NOT USE eth1. The only interface in our HA clusters that talks BGP by default is eth0).

Enable Branch to Branch on the Route Server.

Make sure that both members on the CloudGuard cluster have a default route defined on your internal static routes table.

Create static route tables pointing to the ARS IPs (10.111.0.91/32 and 10.111.0.92/32) and make the next hop the first IP of the FRONTEND subnet. 

Since we are peering to eth0, we want the BGP reply to go out of eth0.

Make sure your rule base allows BGP traffic (TCP 179) between the ARS and the Cluster members

Sample Clish BGP Configuration:

set as 65100                 <— Pick an unused AS number for the Check Points to use (65100 is an example)

set router id 10.111.16.7          <- For an HA cluster the router ID is always the cluster VIP

set bgp external remote-as 65515 peer 10.111.0.91 on

set bgp external remote-as 65515 peer 10.111.0.91 multihop on

set bgp external remote-as 65515 peer 10.111.0.92 on

set bgp external remote-as 65515 peer 10.111.0.92 multihop on

set routemap bgp_export2ARS id 18 on

set routemap bgp_export2ARS id 18 allow

set routemap bgp_export2ARS id 18 match protocol static

set routemap bgp_export2ARS id 18 match network 0.0.0.0/0 exact

set routemap bgp_export2ARS id 18 action nexthop ip 10.111.18.250

set bgp external remote-as 65515 peer 10.111.0.91 export-routemap bgp_export2ARS preference 3 on

set bgp external remote-as 65515 peer 10.111.0.92 export-routemap bgp_export2ARS preference 3 on

set routemap bgp_importARS id 19 on

set routemap bgp_importARS id 19 allow

set routemap bgp_importARS id 19 match neighbor 10.111.0.91 on

set routemap bgp_importARS id 19 match neighbor 10.111.0.92 on

set routemap bgp_importARS id 19 match protocol bgp

set routemap bgp_importARS id 19 action nexthop ip 10.111.18.37  <— The next hop is whatever the first IP of the CloudGuard backend subnet is 

set bgp external remote-as 65515 import-routemap bgp_importARS preference 2 on

[Ave_Joe]

Thanks for the guidance.  I missed the static-route step when I attempted this earlier.  I have a maintenance window to get BGP operational so I hope it goes well.

<-

Hopefully, it will be terminating on the 10.111.0.0/24 VNET. Otherwise, you might be painting yourself into a corner.

<-

Yes.  That is the plan.

[Ave_Joe]

I am sharing the process the was followed to get it working.  I added 3 steps.

1. Make sure Propagate routes is disabled on the Frontend subnet.
2. The AVS admin portals subnet does not learn the 0.0.0.0 route so additional route map entries are needed for any (on-premise) networks that the AVS portals need to be accessed from.
3. Need to add static routes on each cluster member in order to advertise the additional route map entries.

Thanks to Christian and Josh for their help getting everything sorted and working.

<------------

Check Point configuration

1. Check Propagate gateway route setting
a. Check the Frontend network route table configuration to ensure that "Propagate gateway routes" is set to No.

2. Policy update for BGP traffic.

# <- Make sure your rule base allows BGP traffic (TCP 179) between the ARS and the Cluster members
# <- create route server objects and create rule
src: <10.111.0.68> and <10.111.0.69> dst: <10.111.16.7> and <10.111.16.5> and <10.111.16.6> service: <bgp> Action: <allow>

Clish BGP Configuration on both cluster nodes
3. Add static routes for BGP

# <- Make sure that both members on the CloudGuard cluster have a default route defined on your internal static routes table.
# <- Create static route tables pointing to the ARS IPs (10.111.0.68/32 and 10.111.0.68/32) and make the next hop the first IP of the FRONTEND subnet (10.111.16.1).
# <- Since we are peering to eth0, we want the BGP reply to go out of eth0.
set static-route 10.111.0.68/32 nexthop gateway address 10.111.16.1 on
set static-route 10.111.0.69/32 nexthop gateway address 10.111.16.1 on

4. Set AS number

# <— Pick an unused AS number for the Check Points to use
set as 65100

5. Set cluster router ID

# <- For an HA cluster the router ID is always the cluster VIP
set router id 10.111.16.7

6. Configure BGP to peer with Azure route server

# <- route server IP 1
set bgp external remote-as 65515 peer 10.111.0.68 on
set bgp external remote-as 65515 peer 10.111.0.68 multihop on

# <- route server IP 2
set bgp external remote-as 65515 peer 10.111.0.69 on
set bgp external remote-as 65515 peer 10.111.0.69 multihop on

7. Add BGP route map for 0.0.0.0 and all internal private networks on both Cluster members.
All internal private networks are needed because the AVS admin portals do not learn the
0.0.0.0 route as per the documentation.
https://learn.microsoft.com/en-us/azure/azure-vmware/architecture-network-design-considerations

# <- Tells BGP peers that the route to 0 0.0.0.0 and other required networks is 10.111.18.250
set routemap bgp_export2ARS id 19 on
set routemap bgp_export2ARS id 19 allow
set routemap bgp_export2ARS id 19 match protocol static
set routemap bgp_export2ARS id 19 match network 0.0.0.0/0 exact
set routemap bgp_exportARS id 19 match network 10.0.0.0/8 exact
set routemap bgp_exportARS id 19 match network 172.116.0.0/12 exact

# <— The next hop is whatever the IP of the CloudGuard backend LB
set routemap bgp_export2ARS id 19 action nexthop ip 10.111.18.250
set bgp external remote-as 65515 peer 10.111.0.68 export-routemap bgp_export2ARS preference 2 on
set bgp external remote-as 65515 peer 10.111.0.69 export-routemap bgp_export2ARS preference 2 on

8. Add BGP route map for getting to learned networks

# <- Tells BGP that the first hop IP of learned networks is first IP of the back subnet
# <- Needed so traffic to the learned AVS subnets goes out the backend subnet and not the frontend subnet
set routemap bgp_importARS id 59 on
set routemap bgp_importARS id 59 allow
set routemap bgp_importARS id 59 match neighbor 10.111.0.68 on
set routemap bgp_importARS id 59 match neighbor 10.111.0.69 on
set routemap bgp_importARS id 59 match protocol bgp
# <— The next hop is whatever the first IP of the CloudGuard backend subnet is
set routemap bgp_importARS id 59 action nexthop ip 10.111.18.1
set bgp external remote-as 65515 import-routemap bgp_importARS preference 3 on

9. Add static routes on each cluster member so the additional advertised routes can be learned by AVS

# <- Set to the first hop of the CloudGuard Frontend subnet
set static-route 10.0.0.0/8 nexthop gateway address 10.111.16.1 on
set static-route 172.116.0.0/12 nexthop gateway address 10.111.16.1 on

Azure BGP configuration

1. Route server configuration
a. Enable Branch to Branch on the Route Server.

2. Establish BGP with Check Point HA cluster members
# <- Use the physical IP addresses of each cluster node.
# <- First cluster node
Name: nodeHA1 ASN: 65100 IP: 10.111.16.5
# <- Second cluster node
Name: nodeHA2 ASN: 65100 IP: 10.111.16.6