Problem with MAC address on the wrong interface

[fmalfanti]

Hi,

I've a problem with udp packets (dns queries). I'm testing in a simple configuration.

interface eth4 external,  MAC address 00:1c:7f:32:04:21

corrisponding external router interface, MAC Address c4:7d:4f:d6:55:e1

internal interface eth7,  MAC address 00:1c:7f:32:04:1e

internal client IP Address xxx.xxx.xxx.107, MAC Address 00:0c:29:7f:e1:78

when I do a dns query on 8.8.8.8 I see:

---------------------------------------------

> tcpdump -n -e -i eth7 host 8.8.8.8 and port 53 (INTERNAL INTERFACE)

10:09:34.756268 00:0c:29:7f:e1:78 > 00:1c:7f:32:04:1e, ethertype IPv4 (0x0800), length 70: xxx.xxx.xxx.107.40518 > 8.8.8.8.domain: 30244 [1au] NS? . (28)

---------------------------------------------

tcpdump -n -e -i eth4 host 8.8.8.8 and port 53 (EXTERNAL INTERFACE)

10:09:34.756527 00:1c:7f:32:04:21 > c4:7d:4f:d6:55:e1, ethertype IPv4 (0x0800), length 70: xxx.xxx.xxx.107.40518 > 8.8.8.8.domain: 30244 [1au] NS? . (28)

-------------------------------------------------

and everything is OK

Problems start with the answer:

---------------------------------------------

tcpdump -n -e -i eth4 host 8.8.8.8 and port 53 (EXTERNAL INTERFACE)

10:09:34.766040 c4:7d:4f:d6:55:e1 > 00:1c:7f:32:04:21, ethertype IPv4 (0x0800), length 567: 8.8.8.8.domain > xxx.xxx.xxx.107.40518: 30244$ 14/0/1 NS j.root-servers.net., NS k.root-servers.net., NS b.root-servers.net., NS i.root-servers.net., NS c.root-servers.net., NS g.root-servers.net., NS d.root-servers.net., NS e.root-servers.net., NS f.root-servers.net., NS h.root-servers.net., NS a.root-servers.net., NS m.root-servers.net., NS l.root-servers.net., RRSIG (525)

OK, is external

----------------

BUT I SEE THE SAME MAC ADDRESS ON THE INTERNAL INTERFACE

> tcpdump -n -e -i eth7 host 8.8.8.8 and port 53 (INTERNAL INTERFACE)

10:09:34.766053 00:1c:7f:32:04:21 > c4:7d:4f:d6:55:e1, ethertype IPv4 (0x0800), length 567: 8.8.8.8.domain > xxx.xxx.xxx.107.40518: 30244$ 14/0/1 NS j.root-servers.net., NS k.root-servers.net., NS b.root-servers.net., NS i.root-servers.net., NS c.root-servers.net., NS g.root-servers.net., NS d.root-servers.net., NS e.root-servers.net., NS f.root-servers.net., NS h.root-servers.net., NS a.root-servers.net., NS m.root-servers.net., NS l.root-servers.net., RRSIG (525)

HERE MAC ADDRESS ARE WRONG 

then the internal client do a new request

10:09:39.756179 00:0c:29:7f:e1:78 > 00:1c:7f:32:04:1e, ethertype IPv4 (0x0800), length 70: xxx.xxx.xxx.107.52567 > 8.8.8.8.domain: 39751 [1au] NS? . (28)

and MAC ADDRESS in answer are correct

10:09:39.766554 00:1c:7f:32:04:1e > 00:0c:29:7f:e1:78, ethertype IPv4 (0x0800), length 567: 8.8.8.8.domain > 80.86.52.107.52567: 39751$ 14/0/1 NS a.root-servers.net., NS b.root-servers.net., NS c.root-servers.net., NS d.root-servers.net., NS e.root-servers.net., NS f.root-servers.net., NS g.root-servers.net., NS h.root-servers.net., NS i.root-servers.net., NS j.root-servers.net., NS k.root-servers.net., NS l.root-servers.net., NS m.root-servers.net., RRSIG (525)

and the client receive the answer.

No proxy arp, no cluster

ANY IDEA ? 

fabrizio

[PhoneBoy]

What version/JHF?
Also this sounds like it should be a TAC case.

[Timothy_Hall]

As it turns out the Check Point product code does do some caching of MAC addresses in various state tables (which surprised me a bit as maintaining the ARP cache is typically a Gaia OS function); just ran into this gem the other day which drove me crazy for awhile trying to figure it out:

sk116453: IPSec VPN NAT-T traffic is sent to the MAC address of wrong next hop or old IP

Your case sounds very similar to this one:

sk116160: Computers with dynamically assigned IP addresses are not able to access web sites by their...

See if disabling SecureXL temporarily solves the problem if you are not running at least R80.30 Jumbo HFA Take 76+ which contains the fix.

[_Val_]

MAC caching with SXL is okay, if you want to forward accelerated traffic. I am surprised you are surprised 🙂

[Timothy_Hall]

It wasn't the MAC caching itself that surprised me but how it manifested itself.  IKEv2 negotiation would succeed just fine with the Cisco peer gateway (because that is handled outside SecureXL in process space by vpnd).  When the first IPSec NAT-T packets would start to be sent by SecureXL I could see them leaving on the external interface with tcpdump, but the Cisco end said they weren't getting anything.  Bounced IKE/IPSec SAs with vpn tu a couple of times which had no effect, and frankly SecureXL really should have let go of that cached MAC address when I did that.  Even tried failing the cluster over which had no effect either since the MAC cache table is sync'ed between the cluster members.

Finally decided to use -e option to tcpdump to look at the MAC addresses on the IPSec traffic, and I'm scratching my head trying to figure out where this bogus destination MAC is coming from, which is the reason why the IPSec traffic isn't going anywhere.  It's not in the Gaia ARP cache, so I finally searched SecureKnowledge and there was the solution...

[fmalfanti]

I saw  other cases, effects are similar, not the contest. In my case everything is static, no NAT, no VPN. 

arp -an show correct associations MAC/IP.