nic down after upgrading dell r640 to gaia r81

[andy_currigan]

we have 2x r640 dell with x710 intel NIC on a cluster xl r80.30

we upgraded the standby member to r81 successfully

we then upgrade the other member to r81 but the after the upgrade it did not answer anymore to any interface... on the switch all the interfaces are down.

we checked also with the bus information as for sk86121 and the .rules file seems correct even if it's different then on r80.30 and from the other r81 gateway.

both gateway are exactly the same in terms of hardware and firmware revision

any suggestions?

thanks

[Chris_Atkinson]

Are you using the same SFP transcievers or DAC cable at both sites?

[andy_currigan]

yes same cables, furthermore not only the 10gb interface are down but also the 1gb copper interfaces

[Chris_Atkinson]

Thanks for clarifying, if it were limited to the X710 interfaces then I would say sk163267 might have been applicable.

Other customers have had success with the following on the Cisco side in similar cases but likely not here (given your prior involvment in the threads below).

#no lldp transmit
#no lldp receive

Refer also: 

https://community.checkpoint.com/t5/General-Topics/Intel-X710-10gbps-NIC-with-Cisco-Nexus-LINK-probl...

https://community.checkpoint.com/t5/Enterprise-Appliances-and-Gaia/R80-30-3-10-Interface-issues/td-p... 

For information specific LLDP configuration is possible as of R81 and higher per:

https://sc1.checkpoint.com/documents/R81/WebAdminGuides/EN/CP_R81_Gaia_AdminGuide/Topics-GAG/LLDP.ht...

[andy_currigan]

problem solved.

I simply overwrite the existing /etc/ude/rules.d/00-OS-XX.rules that was different then before the upgrade with old one and the interfaces are now back online.

anyway thank you for your support

[Arskaz]

Why has this changed? We I upgraded to R80.40 in the past, the old modified 00-OS-XX.rule was migrated automatically. Now it seems, that upgrading to R81.10 replaces that file with a default one. Not very friendly for remote upgrades...

[Adam276]

Thanks for posting this.  I am bringing this back up as I plan a migration of a gateway to R81.20 on new hardware.  The discovered interface order by default was mixed up on open platform on the new hardware that I have.  There are three 4-port intel nics (the first one is a built in 4 port intel Nic).  eth0 and eth1 were detected on the first card,  eth2 and eth3 were assigned to the last two in the second nic and then the rest are shifted accordingly

This is how they were detected in the old hardware...

0 1 2 3     7 6 5 4    11 10 9 8

This is how they are detected in the new hardware...
0 1 4 5     3 2 11 10     9 8 7 6

I fixed them so that they are the same order using the 00-OS-XX.rule to make it less confusing during the upgrade but if at any point during a future CPUSE upgrade they could default back to the mixed up order,  I would rather just keep the mixed up order to minimize problems with upgrades.  I am doubting now my decision to reorder them and thinking about changing them back to the detected order before deploying the new hardware.

This is a cluster and both new gateways were detected in the mixed up order so it wasn't just a one time thing with the mixed up ordering.

Did you find out if that is the norm going forward for CPUSE upgrades to not migrate the 00-OS-XX.rule file?  That would definitely complicate remote site upgrades if they don't and the NIC ordering changes on the reboot of the CPUSE upgrade.  Did you use blink CPUSE upgrade or the normal upgrade method in CPUSE?  Maybe there is a difference there.  I would have expected that important file to always be migrated though regardless.