Add routes to Azure Scale Set

[J_Saun]

We have a scale set that was created in Azure. Unfortunately I do not have access to Azure. How are routes added to the scale set? Through the Gaia gui or via the Azure interface?

[PhoneBoy]

Ultimately, in IaaS settings, the routing is controlled by the underlying 'aaS' platform (in this case Azure).
There are some routes on the instance but the most you can influence there is what interface the traffic might go out.
Where it goes from there is controlled by Azure, fundamentally.

[J_Saun]

Ok. Right now I have a large route (10.0.0.0/8) on the scale set pointing to the internal load balancer. That should be carved up and have some of the 10.x.x.x (lets say 10.1.1.0/24) pointing towards the internal load balancer and some (10.30.1.0/24) pointing to the external load balancer. Would this routing need to be updated in Azure?

[PhoneBoy]

Yes, these are done as User Defined Routes in Azure.
Refer to: https://sc1.checkpoint.com/documents/IaaS/WebAdminGuides/EN/CP_VMSS_for_Azure/Content/Topics/Overvie...

[Matthias_Haas]

by

<Right now I have a large route (10.0.0.0/8) on the scale set pointing to the internal load balancer

do you mean the route on a ScaleSet Checkpoint  member?

If so, the next hop should be the first IP of the internal subnet (where eth1 is connected to) and not the internal loadbalancer IP

With the latest ScaleSet template Checkpoint is adding at least four routes  per default:

- rfc1918 IPs (10/8,172.16/12, 192.168/16) pointing to the first IP of the internal network (eth1)

- default route pointing to the first IP of the external network (eth0)

That is ok and in most cases you do not have to make any modifications 

Another story is to get the traffic to the scale set

In this case you need UDRs which have the VIP of the internal loadbalancer as the next hop. This is done completely in Azure.

How such a UDR looks like depends on how your VNET and subnet are designed, what  Peerings you have and which traffic you´d like to forward to the scaleset

[HeikoAnkenbrand · Accepted solution]

Hi @J_Saun 

You must set multiple routes on the Check Point cluster gateway and routes in azure.

You may need to adjust your networks accordingly

Example:

           (1.1.1.1) frontend-lb     <>            check point gateway             <>   backend-lb (10.0.1.4)         <>        example network 10.0.2.0/24

azure network controler 10.0.0.1                                                     azure network controler 10.0.1.1      

Check Point gateway:

0.0.0.0/0 to your external network 10.0.0.1

10.0.0.0/8 to your internal network 10.0.1.1

Azure routing UDR example for a network 10.0.2.0/24 behind the gateway:

0.0.0.0/0 virual appliance 10.0.1.4 (IP address of the backend-lb)

10.0.0.0/8 virtual appliance 10.0.1.4 (IP address of the backend-lb)

10.0.2.0/24 virtual network (your example 10.0.2.0/24 network)

[J_Saun]

Thanks for all of the replies. Do I leave the default routes that were added (the large RFC1918's) and just add my user defined routes (UDR's) in Azure? Will the more discrete UDR's be used first?

[PhoneBoy]

UDRs work on the same principle routes on a traditional PC or router work: the most explicit route (with the highest priority) wins.

[J_Saun]

Before I add UDR's I wanted to validate something. Please reference the attached drawing (Note - address space depicted in the diag is not actual but mimics the setup we have),

My Scale set route table currently looks like this:

Default points to 10.70.80.1

10.0.0.0/8 points to 10.200.200.10

172.16.0.0/12 points to 10.200.200.10

192.168.0.0/16 points to 10.200.200.10

As you can see in the diag the External Load Balancer has an Internet IP on it (52.200.100.9) yet my default route points to a host on the external network 10.70.80.0/24. The Azure team says that 10.70.80.1 is an IP owned by Microsoft Azure.

Shouldn't my default route point to 52.200.100.9?

[J_Saun]

Not sure why this has been accepted as a solution. Checkpoint support said 'you dont add routes on the firewalls, you add them in Azure as UDR's' but did not detail WHERE to add them in Azure. Do we add them on the load balancers or the firewall?

[HeikoAnkenbrand]

Hi @J_Saun 

You must set routes on the check point in direction of the internal Azure network. If the support wrote something else it is wrong. 

Load Balancer: On the Load Balancer you have to set no routes and you can't. Only for incoming NAT from the Internet, load balancer rules must be set on the frontend-lb.

Firewall: Set routes to internal network.

Azure: Set UDR on internal networks to backend-lb IP and set 0.0.0.0/0 (default route) on internal networks to backend-lb IP 

Here an cutout from sk110194 - Deploying a Check Point Cluster in Microsoft Azure 

CUT>>>

Setting up routes on the cluster members to the Internal subnets

SSH into each of the cluster members and add the following route: 

clish -c 'set static-route VIRTUAL-NETWORK-PREFIX nexthop gateway address ETH1-ROUTER on' -s

Where:

• VIRTUAL-NETWORK-PREFIX is the prefix of the entire virtual network (e.g. 10.0.0.0/16)
• ETH1-ROUTER is the first unicast IP address on the subnet to which eth1 is connected (e.g. 10.0.2.1)

For example: clish -c 'set static-route 10.0.0.0/16 nexthop gateway address 10.0.2.1 on' -s

Note: If the virtual network is comprised of several non-contiguous address prefixes, repeat the above for each prefix.

<<<CUT

[J_Saun]

Ok. But this is a scale set, currently running at 2 instances (to a maximum scale of 8). If I modify the routes on these 2 instances will those routes get automatically propagated to the additional scale set instances once they come online?

[Matthias_Haas]

this section of the SK is outdated/no longer necessary:

<Setting up routes on the cluster members to the Internal subnets

With the latest ScaleSet template routes for the RFC1918 IP ranges  are added per default, no modification is necessary.

[J_Saun]

Thanks. I do see those RFC1918 routes on the scale set in the Gaia gui.

We have a need to route to an RFC1918 address outside (externally) through the expressroute gateway to an on prem environment which I why I am asking how to configure UDR's. I just realized that I failed to mention this in my original post. My apologies.

Right now when I try to go form the manager to this external RFC1918 address, it hit's the scale set and then returns right back out the same interface in which it came in so I need tell them Azure team where to put this RFC1918 route (scale set or load balancer or both)

[Matthias_Haas]

you need two UDRs:

one for the subnet in which your manager is located:

external rfc1918 network ->(next hop)  internal LB VIP

I guess you already do have this in place as the packet is reaching the scale set.

Second UDR is for the subnet in which your expressroute gateway is located (called GatewaySubnet) :

manager Subnet -> (next hop) internal LB VIP

You do not need a UDR for the subnet(s) in which the scalest is deployed.

Is is normal that only the internal interface (eth1) is used, it´s like a  OneArmed setup.

So the packet flows looks like this:

manager --->VIP LB--- (eth1)--> scaleset member --(eth1) --> express router ----> destination

destination ----> express router ---> VIP LB ---(eth1) -->  same scaleset member --(eth1)--> manager 

[J_Saun]

Thanks. I've updated the diagram to reflect what I think needs to be done based on your reply.

Summary:

- Add a UDR in the External Network vNet - DEST=10.20.30.0/24 - Next Hop=Express Route GW

- Add a UDR in the External Network vNet - Dest=10.90.80.0/24 - Next Hop=10.200.200.10 (Internal LB VIP)

Does this look correct?