Site2Site Routing and default route

[eliadr]

Hi all.

I'm struggling with a weird situation.

I've inherited a network.
1 Dc, 1 DR, 10 remote sites.
DC + DR has a 3 FWs cluster (15600), and each remote site has 2 FWs cluster (3200).
We have 2 separate L2 connections between all sites, and Site2Site IPSec VPN on top of that.

Each remote site has static routes as follows:

set static-route default nexthop gateway address <DC Cluster VIP - SDH1> on
set static-route default nexthop gateway address <DC Cluster VIP - SDH2> on
set static-route <FW MGMT network> nexthop gateway address <DC Cluster VIP - SDH1> priority 2 on
set static-route <FW MGMT network> nexthop gateway address <DC Cluster VIP - SDH2> priority 1 on

The DCs has the following static routes to the remote sites:

set static-route <Remote FW network - Internal> nexthop gateway address <Remote Cluster VIP - SDH1> priority 2 on
set static-route <Remote FW network - Internal> nexthop gateway address <Remote Cluster VIP - SDH2> priority 1 on
set static-route <Remote FW 1 - Internal> nexthop gateway address <Remote Cluster VIP - SDH1> priority 4 on
set static-route <Remote FW 1 - Internal> nexthop gateway address <Remote Cluster VIP - SDH2> priority 3 on
set static-route <Remote FW 2 - Internal> nexthop gateway address <Remote Cluster VIP - SDH1> priority 4 on
set static-route <Remote FW 2 - Internal> nexthop gateway address <Remote Cluster VIP - SDH2> priority 3 on

The DCs also has a default route that points to our partners DC.

Once I remove this default route i lose all communication the the LANs in my remote sites.

If I do one of the following:

1. Add a default route to our backbone (he does only L2, and has one IP for management).
2. Add a static route for each remote site with "next hop logical".

everything is working.

I've read and reread all the relevant info I could find, but I still don't get it...

Any insights?

[PhoneBoy]

What do the routes look like when you set them via Next Hop Logical?
Highly encourage a TAC case here.

[eliadr]

I only did some tests with it, but haven't implemented it network-wide.
I thought I'm missing something in the manuals\guides\BPs...

Anyway, it looked like that (at the DC FWs - the branches remained unchanged):

set static-route <Remote site internal network> nexthop gateway logical bond1.<SDH1 VLAN> on
set static-route <Remote site internal network> nexthop gateway logical bond1.<SDH2 VLAN> on

I'm also trying to open a case with Checkpoint, but I'm dependent on my retailer...

Thank you very much for trying to help.

[eliadr · Accepted solution]

So, apparently it was some misunderstanding on our side of how and when VPN routing\regular routing happens.
It wasn't clear enough for us from the documentation.
Also, now I see, my description above wasn't accurate enough - sorry about that...

In short, we had routes from the DCs FWs only to the branches FWs subnets.
We added static routes to all other internal subnets in each branch, and it worked.
TAC explained that there has to be a regular routing decision first, and only then VPN routing kicks in and take precedence.

Thanks for the help.

[Vladimir]

Unless I am not getting it, it looks like your backbone by default does not share CAM tables universally.

So when you add the default route, to the management IP, there is probably arp cache being populated that is accessible to all.

When you are doing "next hop logical", you are just throwing the packets out of the interface without requiring knowledge of the peer's MAC addresses.