Hi everyone,
I’m currently working on a scenario in Azure where I have two Check Point ClusterXL (HA) deployments in different VNETs that need to establish eBGP routing between them.
Environment overview
cpfwr1fwvpn1: 10.11.0.0/22
cpfwr1fwhub1: 10.11.4.0/22
Connectivity: Azure VNET Peering
Cluster IP layout:
cpfwr1fwhub1
VIP: 10.11.4.6
Members: 10.11.4.4, 10.11.4.5
Internal: 10.11.5.5, 10.11.5.6
cpfwr1fwvpn1
VIP: 10.11.1.6
Members: 10.11.1.4, 10.11.1.5
Internal: 10.11.2.5, 10.11.2.6
Initial BGP configuration:
I configured BGP between the private VIPs on the external interface (eth0):
cpfwr1fwhub1
set bgp external remote-as 64598 on
set bgp external remote-as 64598 peer 10.11.1.6 on
cpfwr1fwvpn1
set bgp external remote-as 64597 on
set bgp external remote-as 64597 peer 10.11.4.6 on
Initial behavior (BEFORE changes)
BGP state: Idle
Investigation findings:
Using ip route get:
cpfwr1fwhub1
10.11.1.6 via 10.11.4.1 dev eth0 src 10.11.4.4
10.11.1.4 via 10.11.5.1 dev eth1 src 10.11.5.5
cpfwr1fwvpn1
10.11.4.6 via 10.11.1.1 dev eth0 src 10.11.1.4
10.11.4.4 via 10.11.2.1 dev eth1 src 10.11.2.5
Based on the above, I understand that the VIPs were reachable via eth, but member IPs were resolved via eth1.
Changes implemented (AFTER)
To fix asymmetry, i added static routes so that VIP + all cluster member IPs are reached via eth0 only:
cpfwr1fwvpn1
10.11.4.4/32 via 10.11.1.1 dev eth0
10.11.4.5/32 via 10.11.1.1 dev eth0
10.11.4.6/32 via 10.11.1.1 dev eth0
cpfwr1fwhub1
10.11.1.4/32 via 10.11.4.1 dev eth0
10.11.1.5/32 via 10.11.4.1 dev eth0
10.11.1.6/32 via 10.11.4.1 dev eth0
Current behavior (AFTER changes)
BGP state: Active / OpenConfirm
When reviewing the routing logs, I notice that, i realized that even though the peer is configured against the VIP, but the firewall receives connections from member IPs:
cpfwr1fwvpn1
May 15 16:19:04.729369 [routed] WARNING: bgp_get_open(3143): peer 10.11.4.4+22960 (proto) has provided 4 Byte AS 64597
May 15 16:19:04.729369 [routed] NOTICE: bgp_pp_recv(4215): dropping peer 10.11.4.4+22960 (proto), no such peer configured locally
May 15 16:19:04.729369 [routed] WARNING: NOTIFICATION sent to 10.11.4.4+22960 (proto): code 2 (OpenMessageError) data
May 15 16:19:22.182290 [routed] WARNING: bgp_get_open(3143): peer 10.11.4.6 [eBGP AS 64597] has provided 4 Byte AS 64597
May 15 16:19:22.182290 [routed] WARNING: NOTIFICATION received from 10.11.4.6 [eBGP AS 64597]: code 2 (OpenMessageError) data
May 15 16:19:22.182290 [routed] NOTICE: bgp_peer_close(6403): closing peer 10.11.4.6 [eBGP AS 64597], state is 5 (OpenConfirm)
May 15 16:21:32.732262 [routed] WARNING: bgp_get_open(3143): peer 10.11.4.4+63906 (proto) has provided 4 Byte AS 64597
May 15 16:21:32.732262 [routed] NOTICE: bgp_pp_recv(4215): dropping peer 10.11.4.4+63906 (proto), no such peer configured locally
May 15 16:21:32.732262 [routed] WARNING: NOTIFICATION sent to 10.11.4.4+63906 (proto): code 2 (OpenMessageError) data
May 15 16:21:50.185917 [routed] WARNING: bgp_get_open(3143): peer 10.11.4.6 [eBGP AS 64597] has provided 4 Byte AS 64597
May 15 16:21:50.185917 [routed] WARNING: NOTIFICATION received from 10.11.4.6 [eBGP AS 64597]: code 2 (OpenMessageError) data
May 15 16:21:50.185917 [routed] NOTICE: bgp_peer_close(6403): closing peer 10.11.4.6 [eBGP AS 64597], state is 5 (OpenConfirm)
May 15 16:24:00.739103 [routed] WARNING: bgp_get_open(3143): peer 10.11.4.4+64418 (proto) has provided 4 Byte AS 64597
May 15 16:24:00.739103 [routed] NOTICE: bgp_pp_recv(4215): dropping peer 10.11.4.4+64418 (proto), no such peer configured locally
cpfwr1fwhub1
May 15 14:16:41.361524 [routed] WARNING: bgp_get_open(3143): peer 10.11.1.6 [eBGP AS 64598] has provided 4 Byte AS 64598
May 15 14:16:41.361524 [routed] WARNING: NOTIFICATION received from 10.11.1.6 [eBGP AS 64598]: code 2 (OpenMessageError) data
May 15 14:16:41.361524 [routed] NOTICE: bgp_peer_close(6403): closing peer 10.11.1.6 [eBGP AS 64598], state is 5 (OpenConfirm)
May 15 14:16:58.815991 [routed] WARNING: bgp_get_open(3143): peer 10.11.1.4+3469 (proto) has provided 4 Byte AS 64598
May 15 14:16:58.815991 [routed] NOTICE: bgp_pp_recv(4215): dropping peer 10.11.1.4+3469 (proto), no such peer configured locally
May 15 14:16:58.815991 [routed] WARNING: NOTIFICATION sent to 10.11.1.4+3469 (proto): code 2 (OpenMessageError) data
May 15 14:19:09.367861 [routed] WARNING: bgp_get_open(3143): peer 10.11.1.6 [eBGP AS 64598] has provided 4 Byte AS 64598
May 15 14:19:09.367861 [routed] WARNING: NOTIFICATION received from 10.11.1.6 [eBGP AS 64598]: code 2 (OpenMessageError) data
May 15 14:19:09.367861 [routed] NOTICE: bgp_peer_close(6403): closing peer 10.11.1.6 [eBGP AS 64598], state is 5 (OpenConfirm)
May 15 14:19:26.820545 [routed] WARNING: bgp_get_open(3143): peer 10.11.1.4+45477 (proto) has provided 4 Byte AS 64598
May 15 14:19:26.820545 [routed] NOTICE: bgp_pp_recv(4215): dropping peer 10.11.1.4+45477 (proto), no such peer configured locally
Given all of the above, could you please let me know if the configuration being set up is incorrect? Is any specific NAT required? Could this be related to a scenario that is not compatible with Check Point?
Best regards.
