R80.30 3.10 Interface issues?

[Brian_Deutmeyer]

Has anyone had issues with interfaces flapping on R80.30 3.10?  I have two Intel x710 nic cards installed on an HPE G10, where the ports from one card are in bond1 and ports from the other are in bond2. I was running R80.20 3.10, where everything was functioning. I upgraded to .30 and my ports on bond2 started flapping.  I took the ports out of the bond and destroyed the PO on the switch, but the ports continue to flap. I’m hoping I’m missing something, but I’ve noticed weird things like the lacp-rate not honoring the clish setting (I set fast, but the bond shows slow), auto negotiate not honoring the clish setting (I set auto, ethtool says autoneg not supported), rx/tx ringsize not honoring the clish setting (I increased it, but ethtool still shows default) and the default multiqueue setting does not match for all ports (the working 10G ports have more CPU allocated than should be). I have a ticket open, but was curious if others have this issue.  I also installed JHF50 (ongoing take), but no dice.  I tried this on two boxes, same result. I noticed the i40e driver was upgraded when I went to .30 3.10, so I tried to use the older version from .20 3.10, but no luck. The driver seems OK based on internet searches, but Check Point documentation says the i40e driver is for 40G nics and not 10G. Maybe the driver should be the ixgbe driver, but 80.20 3.10 also used the i40e driver. Thoughts?

[Magnus-Holmberg · Accepted solution]

We needed to add

no lldp transmit

no lldp receive

on our cisco nexus switches to stop link flapping.
Will edit the post tomorrow with more info about it 🙂

[Brian_Deutmeyer]

You are a wizard!  Adding those commands stopped the link flapping!  Do you know why 80.30 3.10 needs those commands?

[Timothy_Hall]

If the Cisco doesn't see the LLDP it is expecting in a timely fashion (or if it is not formatted in a way the Nexus is expecting) that can cause the interface flap.  The support status of LLDP in Gaia is a bit unclear, as the first SK below says that Gaia does not support it, but the second SK (which admittedly is for Scalable Platforms) says the i40e driver sends LLDP anyway:

sk117676: Is Link Layer Discovery Protocol (LLDP) supported on Gaia?

sk135772: Advantech MAC ARP entries are created on an adjacent layer-2 device which is directly conn...

[Brian_Deutmeyer · Accepted solution]

Just a quick update on this topic.  This seems to be related to the Intel X710 chipset on NIC cards.  You can check this by running the lspci command:

[Expert@myFW:0]# lspci | grep X710
12:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01)
12:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01)

Instead of disabling LLDP on the switch side, you can also disable LLDP using ethtool, but note, this does not survive a reboot, so you'll need to add it to /etc/rc.local (I found sleeping for 20 seconds worked well before running the commands), but upgrades/HFs could overwrite this file, which makes it not ideal.

[Expert@myFW:0]# ethtool --show-priv-flags eth0 | grep lldp
disable-fw-lldp        : off
[Expert@myFW:0]# ethtool --set-priv-flags eth0 disable-fw-lldp on
[Expert@myFW:0]# ethtool --show-priv-flags eth0
disable-fw-lldp        : on

That said, I've received a HF for this issue so I don't have to worry about any of this disabling and it it worth asking for the hotfix to see if it works in your environment.  I've also ask this be integrated into the code so a private HF isn't necessary.

One other note to all this is the firmware of the NIC itself.  The ethtool private flag commands and the HF worked on firmware version 10.4.3 for my NIC cards, but I had some NIC cards that had firmware version 5.60 0x8000355f 1.1752.0, which neither the HF or the ethtool private flag commands worked on those interface.  To fix this, I upgraded the NIC firmware using the SPP from HPE so they version was at 10.4.3 and voila.

[Expert@myFW:0]# ethtool -i eth2 | grep firmware
firmware-version: 5.60 0x8000355f 1.1752.0

[JozkoMrkvicka · Accepted solution]

This issue with LLDP is fixed in latest R80.30 Jumbo Take.

More info:

https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solut...