topic Re: weird issue with 40Gbit Module and new 29100 Plus appliances in Firewall and Security Management

weird issue with 40Gbit Module and new 29100 Plus appliances

AleLovaz82 — Thu, 10 Oct 2024 12:01:02 GMT

Hi

we have 6 spare (for now there isnt' any cluster configuration ) 29100 plus appliances ( 81.20 Take 84 ) ,each one is equipped with 1 of this module

2 Port 40GBase QSFP+ / 100GBase QSFP28 interface card for 9000/19000/29000 appliances - CPAC-2-40/100F-D.

Both the port of EACH gateway are connected to a cisco ACI LEAF where all the ports are configured in the same way.

What is my problem?
Only the port2 of EACH modulo goes up.
Is not a Cisco ACI issue or a QSFP+ issue or a fiber issue because we tried to

change port on cisco ACI moving ,and in every port ...module port2 is working,module port1 doesn't.

move all the QSFP and fiber from port2 to port1 , on port2 everything is working fine,in port1 we see only "link down".
Remember that this issue occurs on SIX spare 29100 with SIX different module connected to different leaf on ACI and we tried to change/swap anything

The only difference I can see from CP pov is this one :

[Expert@:0]# ethtool --show-fec eth4-01
FEC parameters for eth4-01:
Configured FEC encodings: Auto
Active FEC encoding: Auto
[Expert@0]# ethtool --show-fec eth4-02
FEC parameters for eth4-02:
Configured FEC encodings: Auto
Active FEC encoding: Off

the "Active FEC encoding" it's indipendent and cannot be changed with " ethtool --set-fec eth4-01 encoding 'parameter' "
Even if I change Fec Encoding to none ,or RB or Baser the "Active FEC encoding" remain Auto in port eth4-01 and Off in port eth4-02

Any idea?

Re: weird issue with 40Gbit Module and new 29100 Plus appliances

Bob_Zimmerman — Thu, 10 Oct 2024 14:17:57 GMT

The forward error correction is negotiated. "Configured FEC encodings" is what you can control. "Active FEC encoding" tells you what value was negotiated, and "Auto" just means it hasn't negotiated a value. It's like the difference in lspci output between LnkCap (Link Capabilities) and LnkSta (Link State; a card which is capable of 8 GT/s and 16x width could be used in a slot supporting only 2.5 GT/s and 4x).

What does 'ethtool -m' show on each interface? Do they both recognize their transceivers? Do both transceivers show received light?

Re: weird issue with 40Gbit Module and new 29100 Plus appliances

AleLovaz82 — Thu, 10 Oct 2024 15:40:03 GMT

ethtool -m eth4-01
Identifier : 0x11 (QSFP28)
holy s**t ,it's the 100gbit trasnceiver. The people in datacenter told us that they used QSFP+ in both the ports.
This should be the issue, thx for the hint , I never head the -m flag on ethtool !
tomorrow we'll see ,someone will change them

Re: weird issue with 40Gbit Module and new 29100 Plus appliances

Bob_Zimmerman — Mon, 21 Oct 2024 20:06:17 GMT

Was that the problem? Seems unlikely six different boxes would all have the wrong transceivers inserted in the same slot and the right transceivers inserted in the same slot.

Re: weird issue with 40Gbit Module and new 29100 Plus appliances

AleLovaz82 — Tue, 22 Oct 2024 07:18:34 GMT

Hi Bob,
wrong SFP in each port 2 of each 29100, and also a cabling issue -.-'