Background:
A Maestro stack with 4 SGM's forming a single security group as a VSX host. 2 interfaces dedicated to the VSX as a vlan trunk configuration:
eth1-09
eth2-09
Virtual system #1 (vsenv 1):
VS created with 4 logical transit interfaces - two east, two west. Using BGP and a default Graceful Restart time of 360s. BFD is currently OFF. These are multiple-path routes to internal/external connected to a Cisco ACI leaf switch which is directly (no multi-hop) connected and a BGP peer to all 4 interfaces.
Transit east:
eth1-09.1 192.168.3.2/30
eth2-09.2 192.168.3.6/30
Transit west:
eth1-09.3 192.168.3.10/30
eth2-09.4 192.168.3.14/30
BFD is OFF.
Graceful Restart is ON and default value is 360s.
We are NOT receiving, and have confirmed that GR is not enabled on the Cisco ACI side (problem), and the Check Point SGM's are not configured for BFD (problem). Everything works great out of the box without tuning; routes shared correctly, multiple paths, multiple interfaces, but where we run into a problem is during Check Point hotfixes and upgrades (or just a reboot of a SGM #1 which handles the BGP peering session as SMO).
Routing table:
[vs1-vsx-ch01-01:1]# clish -c "show route bgp"
B 0.0.0.0/0 via 192.168.3.1, eth1-09.1, cost None, age 12957
via 192.168.3.5, eth2-09.2
CORE TRANSIT
B 10.123.67.0/24 via 192.168.3.9, eth1-09.3, cost None, age 12957
via 192.168.3.13, eth2-09.4
ABC VRF
B 10.213.93.0/27 via 192.168.3.9, eth1-09.3, cost None, age 12957
via 192.168.3.13, eth2-09.4
ABC VRF
To solve an outage issue where the ACI side flushes its routes when the SMO role changes to SGM #2 due to a reboot, it takes about 5-15 seconds to re-establish the BGP peer session with the ACI peers and we create a routing black hole causing a slight outage.
Question:
I am looking to have the ACI enable GR, and on the Check Point enable BFD on each transit interface. A lot of advice I've read is generally speaking you do not want to use both GR and BFD. However, I feel in this particular setup it would be beneficial - if the BGP session manager (SMO) changes during the reboot, the routes don't get flushed immediately (stale timer activated) for 120-360seconds, and BFD doesn't show the link as DOWN during the failover to SGM#2. BUT, if the link phyiscally goes down then BFD would trigger an outage and instruct the stack not to route via that interface since it was now down - therefore failing over ALL traffic to the second link (or vice versa - equal cost routes).
Am I thinking correctly in this scenario? Does BFD take precedence in this situation? Appealing to the other routing guru's in the forum.
Thank you,
Chris
.png "Wolfgang"){kind=avatar}
