VPNs to different sites via different ISPs

[Steven_Sultana]

Hello community!

We're trying to implement the following scenario.

We have several on-premises sites, each with 2 ISPs. We also have a site in Azure (with 1 external interface).

We would like the local sites to have a mesh VPN community which prefers ISP1, and falls back on ISP2. ie, if GatewayX's ISP1 goes down, then the other Gateways bring up a VPN tunnel with GatewayX using ISP2.

We then want that the on-premises sites to have a star community with Azure as the hub, this time preferring ISP2, falling back on ISP1.

I'm trying to mess with combinations of 'load sharing' or 'calculate IP based on network topology' and static routes, but have not had success yet. (I think 'calculate IP based on network topology' does not do exactly what it seems to say on the tin)

All gateways are centrally managed by the same Management.

Did anyone ever configure such a setup?

Thanks!

Steven.

[PhoneBoy]

Did you change the Outgoing Route Selection at all?
It would be helpful if you could explain (along with version/JHF level) the exact combinations you tried with the precise results you got.
Screenshots (with sensitive data masked) might also be helpful.

[Steven_Sultana]

Thank you for the reply.

Version is R80.40, Take 102 for management and gateways.

IPs are currently 'fake' as I'm doing tests in a lab which simulates the end result as much as possible.

ISP1 = 213.165.x.x

ISP2 = 195.158.x.x (this one is behind an interlink, so sometimes you will see 192.168.x.x)

Azure ISP: 13.13.x.x

Routing Config:

GW1 (on prem)

• ISP Redundancy in HA
• Primary: ISP 2 (195.158)
• Secondary: ISP 1 (213.165)
• Apply settings to VPN - DISABLED
• Static routes to GW4 - ISP 1:
• Via ISP 1 – preference 3
• Via ISP 2 – preference 6
• Static routes to GW4 - ISP 2:
• Via ISP 2 – preference 3
• Via ISP 1 – preference 6

GW4 (on prem)

• ISP Redundancy in HA
• Primary: ISP 2 (195.158)
• Secondary: ISP 1 (213.165)
• Apply settings to VPN - DISABLED
• Static routes to GW 1 - ISP 1:
• Via ISP 1 – preference 3
• Via ISP 2 – preference 6
• Static routes to GW1 - ISP 2:
• Via ISP 2 – preference 3
• Via ISP 1 – preference 6

CP-Cluster (Azure)

• Not actually a Cluster.
• 1 ISP (13.13.x.x)
• No special static routes

VPN config:

GW1:

• IP selection by remote peer: Use probing – HA – ISP 1 primary
• Outgoing route selection: Route based probing
• Source IP address setting: Automatic

GW4

• IP selection by remote peer: Use probing – HA – ISP 1 primary
• Outgoing route selection: Route based probing
• Source IP address setting: Automatic

CP-Cluster

• selection by remote peer: Selected from topology table: 13.13.x.x
• Outgoing route selection: OS routing table
• Setup: use outgoing traffic configuration
• Source IP address setting: Automatic

Results:

For GW1-to-GW4 and GW4-to-GW1, the tunnel is up and both ends are using ISP1.

Then for CPGW01-to-CPCluster, the next-hop IP is 192.168.x.x, ie ISP2 – cool this is what we want.

But for CPCluster-to-CPGW1, the Peer IP is ISP1. This is confirmed from TCPdumps (taken from GW1. GW1 is talking to 13.13.x.x via 192.168.x.x - ISP2, but CPCluster is talking back to 213.165.x.x - ISP1).

(I will try to post screenshots, but for some reason it's not working yet...)

So the VPN is coming up asymmetrically. This asymmetry is causing issues elsewhere.

We need to somehow inform CPCluster that the peer address should be 195.158.x.x, then fall back to 213.165.x.x if need be.

Should I still go through the different steps I took during testing?

Steven.

[Steven_Sultana]

Screenshots.

[the_rock]

When you say inform the cluster of one IP and then fall back to a different one, Im not sure I understand that, sorry. It will always recognize the same IP and only fall back to another one if there is ISP redundancy configured and primary link goes down. What does link selection page look like, can you share that?

[Steven_Sultana]

Thank you.

I'm attaching the link selection page for GW1, and also attaching a simplified version of the topology.

We want 1 VPN between GW4 and GW1 via ISP1, and 1 VPN between CPCluster and GW1 via ISP2.

But if either ISPs fails, then the respective VPN is brought up with the other ISP.

[the_rock]

So the way you have it now, IP 213.x.x.x will ALWAYS be primary link that VPN will use and should it fail, it would go to secondary link listed. You can also set this up via ISP redundancy tab in gateway properties with option "apply settings to VPN".

[Steven_Sultana]

Indeed. Having the primary link configured is probably not something that I want.

But when I remove that, what's to tell GW4 to prefer connecting to 213.x.x.x and CPCluster to prefer 195.x.x.x?

[the_rock]

Ok ok, I totally get what you are asking now, sorry for the confusion. Short answer is "I have no clue", but, it would be nice to find out if this is even possible. I will let @PhoneBoy  take a "stab" at it, as he is CP encyclopedia, but in the meantime, let me see if there are some settings in trac_client_1.ttm file on the gateway or guidbedit that may help.

[Dan_Cannon]

I very much doubt this complexity would be possible without using advanced routing protocols and VTi interfaces.  If it is possible you'll need to review edits to vpn_route.conf, user.def.FW1,crypt.def and or table.def depending on the requirements?  Also have a look at the SK,s for controlling traffic with ISP redundancy - but given the requirement for "if this tunnel fails then..." this would better suit dynamic routing and VTi's I think...

[the_rock]

I see full logic in what you said, agreed.

[Steven_Sultana]

Hello Dan, still trying to digest your reply.

I think in our case, then, if we go for VTIs, on each gateway we will need:

2 VTIs (for each ISP on the local gateway). and multiply that by 2 (for each ISP on the remote gateway). And do that for each remote gateway.

Right?

[Dan_Cannon]

yes correct, then you can use dynamic routing to define paths/failover requirements.

hope this helps...

Dan