VPN Redundancy without using Dynamic Routing

[JaySon_2021]

We have 3 sites. All of them use Checkpoint firewalls (R82.10). We want to have full redundancy between all sites.

Examples:

- If VPN-A goes down, we want Site1 to go to Site2 to get to Site3, and the reverse (Site3 --> Site2 --> Site1)

- If VPN-B goes down, we want Site1 to go to Site3 to get to Site2, and the reverse

Etc.

We want to accomplish this without using Dynamic Routing (OSPF, BGP) as these are not large sites and we want to simplify supporting it.

Is this possible without using Dynamic Routing? Can it be done using Domain Based VPNs?

[simonemantovani]

In my opinion dynamic routing protocol will simplify your infrastructure my making it more automatic everything.

If you don't want to use route based vpn, but you want to use domain based vpn, you need to manage manually the encryption domain adapting them when a failure occurs (you can't have the network in all the encryption domains for all the firewall in vpn community.

I understand what you write but dynamic routing (BGP) and routed based vpn is the best configuration you can implement.

[JaySon_2021]

Thanks Simonemantovani

What if I just wanted the following 2 scenarios solved:

- If VPN-A goes down, we want Site1 to go to Site2 to get to Site3, and the reverse (Site3 --> Site2 --> Site1)

- If VPN-C goes down, we want Site2 to go to Site1 to get to Site3, and the reverse

If VPN-B goes down between Site1 and Site2, so be it - we wait for it to come back up. As long as Site1 can still get to Site3 (not routing through it to Site2), and Site2 can still get to Site3 (not routing through it to Site1)

Would this be possible, without Dynamic Routing, and not require manual intervention?

[simonemantovani]

Without diynamic routing you need manual intervention to adapt the encryption domain to avoid overlapping encryption domain.

[Bryan-Smith]

@JaySon_2021, If Site1 loses its direct VPN to Site3, traffic can either stop immediately or automatically reroute via Site2. The behavior depends entirely on whether backup static routes are configured. No dynamic routing is required, but bidirectional route design is critical. 

Backup routing in a Check Point Route Based mesh VPN is achieved by pre‑staging higher‑metric static routes over alternate VTIs and relying on tunnel monitoring to withdraw primary routes during failure. 

Check Point's route-based VPN with numbered VTIs makes this feasible because you treat the tunnels like interfaces for static routing. 

As was stated by @simonemantovani this will not work with domain-based VPNs & I would not do this past three sites. Managing all of the static routes in each direction is prone to mistakes. 

[JaySon_2021]

Thanks @Bryan-Smith . Would this be done in a single mesh VPN? Or would each site have a separate Mesh VPN? 

[Bryan-Smith]

@JaySon_2021 - single mesh would be recommended. 

[Bob_Zimmerman]

With route-based VPNs, the community is only used to configure the negotiation parameters, like the IKE version, cryptographic algorithms, and so on. If you want all three links to use the same parameters, you may as well put them in one community.

Also note you cannot use the community in rules, as traffic over a VTI will never match the community object. Write the rules as if you had a really long ethernet cable plugging the firewalls into each other directly with no VPN involved.

[JaySon_2021]

Thanks all. I configured a 3 firewall Route Based VPN in a lab using the simplified method in R82.10 and it was pretty straight forward. I added static routes as follows:

FW_A> show configuration static-route
set static-route 10.1.2.0/24 nexthop gateway address 169.254.0.2 on
set static-route 10.1.3.0/24 nexthop gateway address 169.254.0.5 on
FW_B> show configuration static-route
set static-route 10.1.1.0/24 nexthop gateway address 169.254.0.1 on
set static-route 10.1.3.0/24 nexthop gateway address 169.254.0.3 on
FW_C> show configuration static-route
set static-route 10.1.1.0/24 nexthop gateway address 169.254.0.6 on
set static-route 10.1.2.0/24 nexthop gateway address 169.254.0.4 on

I also set permanent tunnels on the Community object. I am having an issue though. I can only get 2 tunnels up at a time, unless I remove a route from the firewall that isn't working pointing to one of the other firewalls, then all the tunnels will come up and I can add the routes back and the tunnels stay up. But it's random. If I reboot all 3 firewalls then next time one of the other tunnels wont come up. Not sure what is causing this.

Any help would be appreciated.

[simonemantovani]

What is not realy clear for me, but maybe someone more skilled than me could explain, if I look at AZ_fw, this firewall has two point-to-point connection between 169.254.0.6 (TR_fw) and between 169.254.0.4 (STC_fw), but it has not a point to point between the 169.254.0.1, so this route is a little bit ambigous and not working and needed.

Why do you configured it?

[JaySon_2021]

@simonemantovani  - Sorry. My bad. That route was there from when I was t-shooting. I removed it. Still have the issue with one tunnel not coming up.

[simonemantovani]

Ok fine ... that is really weird, did you collected any vpn debug or fw monitor capture, just to have any further idea to what is happening?

[Bob_Zimmerman]

What does an IKE debug tell you? Is the third tunnel even attempted?

[JaySon_2021]

I ran the following 2 commands on the AZ firewall because it cannot ping to 10.66.6.1 which is behind STC_fw

vpn debug trunc
ike debug trunc ALL=5

I get data in both iked0.elg and vpnd.elg, but when I open them in IKE View It just says "IKE Debugging Info" on the left and "Log File" on the right.

Am I not doing the debug correctly?

[Bob_Zimmerman]

I just use 'vpn debug ikeon', but trunc can be okay if you don't want to keep older data for comparison.

Once IKE debugging is enabled, use 'vpn tu' to clear all of the negotiations, then let them renegotiate. That should give you some good data in the files.

[the_rock]

@JaySon_2021 

To also add this, make sure to turn off debugs after:

fw ctl debug -x

fw ctl debug 0

[Bob_Zimmerman]

I never turn off IKE debugging if I can help it. It's very low-volume, and once a VPN is working, old data lets you see what worked before so you can compare it to what's failing when somebody complains about it.

[the_rock]

I could be wrong, but I believe one way to make this work with way you want is to possibly have multiple static routes to destinations you desire, just give them different metrics, so ones with higher metric would only take effect if mai one was to fail.

[JaySon_2021]

Hi Andy,

Yes. I will modify the routing with metrics to get my full redundancy once I figure out this issue with the tunnels not coming up. It seems that with "Permanent Tunnels" set on the Community object I never get all 3 VPN's up. If I disable "Permanent Tunnels" on the community I eventually get all 3 VPN's up. But it takes time and of course interesting traffic.

[the_rock]

I would check these 2 things are set. Set ALL enc domains ae empty groups (just modify rules to include desired subnets) and also, set tunnel mgmt per gateway option.

I never had any issues making it work that way. Only exception was SASE tunnel, where we had to set sase side as main gateways, but if its meshed community, that would not even matter.

[JaySon_2021]

Hi Andy,

Thanks for the suggestions. I did as you said and created an 'EMPTY' Encryption Domain object group for each firewall and applied it. Still no change. I have a TAC opened now, so will hopefully get a resolution to this issue tomorrow. Will update this thread with what is discovered.

Thanks all for your help.

[the_rock]

One more thing...did you set tunnel mgmt "per gateway"?

Never an issue, we are always here to help each other.

[JaySon_2021]

Yes. Sorry. I should have also said that. The default was for 'One VPN per Gateway', and I left it as that. Never changed.

[the_rock]

K, sounds good. Keep us posted what TAC says. Sounds like you got the config corretly set.

[the_rock]

Hey mate,

Any update from TAC?

[JaySon_2021]

Hi Andy,

Prior to sync'ing up TAC I decided to rebuild my Lab environment completely (New SMS, and 3 FWs). I was seeing weird ARP anomalies on the Internet facing interfaces that made me think something was up with the build.

I rebuilt the environment exactly the same, with different 192.168.1.x IPs for the front end because I thought that was where the issue may have lied and was concerned about ARP cache possibly causing me grief if I used the same IPs for the FE (Internet).

Thankfully, after it was all built and up and running, I had solid VPN connectivity between all 3 sites. I rebooted the boxes after each change just to be sure, and still all good.

The only thing that bit me, and I do not understand, was that when I implemented redundancy (priortized routes) at each firewall I was not able to ping from the firewall to the remote firewall (ie: could not ping from TR_fw, to Az_fw, through STC_fw.) where as I was before configuring redundant routes.  I had to ping from an address behind TR_fw (a client with IP 10.66.5.2) in order for the routing from TR to STC to AZ to work.

Thanks to all for their assistance and suggestions. Glad I have this working.

I can provide full details of my build steps and route config if people think that would provide value for others.

Regards,

[Bryan-Smith]

Great to hear! Full details would be awesome for the community.