ARP issue on the firewall

[ashish_verma]

Hello guys,

I was getting network degrade performance complaint for one of the sites and after investigation I observed that seems to be an ARP issue but I am not able to understand why it is happening. Below is the detail-

We are having one checkpoint cluter on site_1, one cluster on site_2 and a fortigate firewall on site 3, all connected in the same subnet 10.1.0.0/24. Site_1 cluster IP is 10.1.0.5. When I am trying to ping from Site_1 IP to Site_3 fortigate firewall and look into the arp table of Forrigate for 10.1.0.5, I saw the arp entry is changing between the mac address of site_1 cluster and site_2 cluster. Site_2 cluster is also getting some ping replies. I checked in the dash board if there is any NAT configured on site_2 for 10.1.0.5 but did not find anything.

Please help with this issue what might be cause of this. Thanks in advance. 

[ashish_verma]

One more thing would like to add that cluster is also showing down on site_2 firewall. 

[Alessandro_Marr]

are you using:

Proxy ARP?

clusterXL high avaliability or load sharing?

VMAC?

[ashish_verma]

I did not see any proxy arp entry in the webUI. But we are using manual NAT so I think it is automatically generated, need to check that. 

The cluster is in HA mode and not using VMAC. 

[Alessandro_Marr]

did you check on both cluster 1 and 2? 

[ashish_verma]

I checked on both the gateways but no manual proxy arp is configured. When I issue fw ctl arp it is showing there. But that is strange because I am not able to see any nat rule for this IP. Also there is no local.arp file is in the directory. What might be cause of this issue and How can I delete this arp entry showing in fw ctl arp? 

[Timothy_Hall · Accepted solution]

I'm assuming the site_1 and site_2 Check Point firewalls are managed by the same SMS.  If so, you have an automatic NAT rule configured in the policy but you forgot to change the "Install On Gateway" from the default of "Any" to either the site_1 or site_2 gateway specifically.  That is why you are seeing the proxy ARP entry in the fw ctl arp output on the "wrong" firewall.  Correct the Install On Gateway field for the automatic NAT referencing 10.1.0.5 and push policy to BOTH sites.

In my CCSA classes, I talk about the effect of leaving the "Install On Gateway" set to Any as helping trigger what I call the NAT Bomb when a second site/gateway is added for the first time.  Most sites start with just one firewall/cluster and they configure NAT as needed.  Unfortunately they leave the "Install On Gateway" set to "Any" in their automatic NAT setups or even in manual NAT rules.  This is fine as long as there is only one gateway/cluster being managed.  However when a second firewall/site is added and is managed by the same SMS, the NAT Bomb is triggered as soon as the policy is installed to the new gateway.  It immediately starts trying to perform all the NATs configured for the original gateway, and will even attempt to proxy ARP for the original firewall's NAT addresses as well in the case of automatic NATs.  This causes instant mayhem at both sites as asymmetric routing occurs and TCP out of state conditions are rampant.  This lively class discussion is further reinforced by all the Automatic NAT example screenshots in the CCSA courseware showing the "Install On Gateway" left set to the default of  "Any".  Grrrr.

--

CheckMates Break Out Sessions Speaker

CPX 2019 Las Vegas & Vienna - Tuesday@13:30

[Bryce_Halkersto]

If you have 2 CP clusters in the same subnet it possibly could be a magic mac issue.  See sk25977.

[ashish_verma]

Both the clusters are having different cluster ID. Does gateway send CCP packet out all the interfaces or only thr sync interface? Because I am getting CCP packets on other interfacez also. 

[PhoneBoy]

CCP packets are sent on all interfaces.