Issues with ARP: (Partially) Not receiving on one interface

[k_b]

Hi,

I am currently facing a strange problem. Atleast its a strange problem for me, I hope it´s not for you, so that you can help :).

Status Quo

Three devices within one VLAN x, device A, device B and device C, while device B and C are in a cluster and device C is the device under investigation.

Device C is connected to this VLAN / network via two interfaces which are forming a bond, let´s say eth1,eth2 are forming bond1.

Situation I

When eth1 and eth2 are up, device C can ping device A and device B => All good.

Situation II

When disabling eth1 and leaving eth2 up, device C can ping device A and device B => All good.

Situation III

When disabling eth2 and leaving eth1 up, after a while (ARP timeout) device C can not ping device B anymore but device A and all other devices within this VLAN.

It can be observed that the ARP entry for device B will change to state "incomplete" after a while on device C. ARP request from device C for device B are visible via tcpdump on device C and B. ARP replies are visible on device B but not on device C. In case the ARP entry for device A is deleted manually on device C you can observe both, APR requests and replies, on device C and device A.

For my understanding it can not be a physical issue. Why should a ping be possible to device A but not to B in case this cable is not working correctly anymore? Why can device C receive ARP replies from device A and all other devices beside of device B? Why does device C receive ARP replies in case the interface (eth2) is up again?

Thanking you in advance

k_b

[PhoneBoy]

What version/JHF level?
How precisely are you disabling the other interface in the bond?

[k_b]

I was hoping this question would not be raised. Currently it is still used the r77.30 (a replacement is currently in perparation).

The interfaces are disable via expert mode and ifdown ethx.

[Chris_Atkinson]

What was the Jumbo hotfix version applied?

Any more details available on the make/model of switch involved?

[cem82]

If I understand the topology/issue, is the bond configured as a vPC across multiple switches?  I'm thinking the vlan may be missing on one of the trunks somewhere in between?

[Chris_Atkinson]

What bond protocol / method is used on the devices each side?

[k_b]

LACP

[k_b]

Yes, the bond is connected to two different switches so a vPC is formed but C (device under investigation) is possible to ping other devices within this VLAN (for example A), only the cluster member (device B) can not be pinged. I think the VLAN should be okay. Also in case the vPC would be confused (so could does not know how to reache device C), no communication at all should be possible for device C within this VLAN. That´s the confusing part for me: What is the reason for the communication between B and C to be "special"?

[Timothy_Hall]

The communication between B& C is special because the bonding protocol is choosing a different physical interface based on the MAC addresses or possibly layer 3&4 values involved.

Connect both physical interfaces of the bond to the same physical switch and repeat your connectivity test.  If it still fails you have an issue with your bond setup.  If it succeeds you have an issue with your vPC/VLANs between switches.

[k_b]

Hi,

sure, I can try it, but for me this would not explain

• why ping (which would select the same interface because of same destination / source mac / IP) is working until ARP is timed out.
• why (atleast I would hope it does not) the mentioned interface should be selected as the communication path in the situation it is shut down.