how to properly run BFD over VSX virtual Switches?

[Thomas_Eichelbu]

Hello Check Mates, 


i have following situation:

we have created a VSX Setup.
5 VS Systems 
they are connected over a virtual switch acting as a "backbone link"
this 5 VS systems connects to a bunch of Cisco Routers and do dynamic routing with OSPF.
we distribute all routes from the Cisco world to the Check Point world and we redistribute default routes to the Cisco Routers.
this works so far.

for fast convergence we use BFD to speed up the OSPF, this works very well.
also we want(ed) to use BFD to communicate between the VSX Systems, but this seems not to work.

output from one the VS looks like this:

show ip-reachability-detection

Ping Count: 3
Ping Interval: 3

BFD Minimum TX Interval: 300 ms
BFD Minimum RX Interval: 900 ms
BFD Detect Multiplier: 3

*Only the cluster master can send or accept ICMP packets.

Remote Address Protocol Reachable*
x.x.x.1 _ _ _ _ _ _ _ _ _ _ BFD (S) Yes
x.x.x.2 _ _ _ _ _ _ _ _ _ _ BFD (S) Yes
x.x.x.81 _ _ _ _ _ _ _ _ _ BFD (S) Yes
x.x.x.82 _ _ _ _ _ _ _ _ _ BFD (S) Yes
x.x.x.105 _ _ _ _ _ _ _ _ _ BFD (S) Yes
x.x.x.106 _ _ _ _ _ _ _ _ _ BFD (S) Yes
y.y.y.210 _ _ _ _ _ _ _ _ _ BFD (S) Unknown <- Check Point VS, it should see at least 5 i see only 2 ?
y.y.y.211 _ _ _ _ _ _ _ _ _ BFD (S) Unknown <- Check Point VS, it should see at least 5 i see only 2 ?

this is configured as BFD Singlehop. i dont got BFD Multihop running. if i choose PING for "ip-reachability-detection"
then it is showing as UP.
Between the VS is just a a flat transit network made via a VSX switch.


we also figured out, if the BFD is configured not the same over all VS the OSPF process is flapping when adding or removing interfaces to a VS which is in fact very dramatic. we deleted and added interfaces on the VS´s via SmartConsole and the OSPF routes got totally lost.
we saw not all BFD settings, were configured equally, some had BFD and some had PING for IP "ip-reachability-detection".
After deleting all BFD configuration between the VS the OSPF routes did not disappear when adding/deleting interfaces to an VS, via SmartConsole ...

Question: what would you do?
BFD between the VS, YES/NO
use PING for IP-REACHABILITY instead of BFD? YES/NO
is it BFD Multihop? YES/NO

software version is of couse the latest and greatest, R81.10 + Take 55

best regards

[Wolfgang]

@Thomas_Eichelbu in the past we tried something similar with probing different VSs as destination from other VSs. It was a nightmare, somtimes working some not, running VSLS and moving active VS from one node to another node results in a desaster.

I don't know how exactly the communication works internal if you have only inter VS traffic but it feels like something "magic" 😕 Debugging such a communication will be too problematic, because you don't see all of the packets on the internal wrp interfaces.

Good luck and hope someone from Check Point can help 😉

[Thomas_Eichelbu]

@Wolfgang, 

i fear you are right. The more i think about this topic the more iam convinced BFD makes me sense between the VS instances over a Virtual Switch since the OSPF has nothing to converge too.
if a VS becomes unavailable it has no second path either to fail over too.
so better to remove the ip-reachability-detection" between the VS and leave it only for the OSPF peers.

Check Point TAC is already working on it ... but more on the issue with the lost OSPF routes when adding/removing interfaces.

best regards