topic Re: 9800 interfaces links down but switchports up in General Topics

9800 interfaces links down but switchports up

genisis__ — Sat, 11 Oct 2025 12:05:06 GMT

Just thought I would post this issue I've just experienced.

9800 appliances with extra interface modules configured as VSX running R82 with JHFA41.

Symptom:

When switchports are enabled which lead to the interfaces on the 9800 it appear that this is up, however on the 9800 side the link status is down. Visually on the switch where the port should be green, this is orange (Note: Onboard interfaces where fine)

Originally I had only one interface showing this issue and after trying different things I powered off/on the appliance via the LOM.

After this, the port started to work normally again, however, now other ports where showing this issue, additionally it was noted that a number of processes had terminated.

I logged a TAC case with Checkpoint (who are still investigating this).
Eventually via the LOM I did a hard reset, and after this everything was good, including the processes. At this point its not clear if this is a hardware issue or a potential undiscovered feature (bug!).

I have 4 units in totally and thus far only seen this on one units, which for me would seem more like hardware issue, but I will wait for Checkpoint to conclude there investigation.

Re: 9800 interfaces links down but switchports up

the_rock — Sat, 11 Oct 2025 20:50:04 GMT

I find usually ifdown ethx; ifup ethx works well.

Andy

Re: 9800 interfaces links down but switchports up

Chris_Atkinson — Sun, 12 Oct 2025 05:03:42 GMT

Which expansion NIC and SFPs out of interest?

Re: 9800 interfaces links down but switchports up

genisis__ — Sun, 12 Oct 2025 19:59:27 GMT

Hi Chris,

TAC came back with this.

If the system is powered off abruptly or rebooted without a clean shutdown, the file system may be marked as "corrupted."
On the next boot, the operating system will check the file system status:

If the file system is marked "corrupted," fsck is automatically invoked during boot.

[ 29.026418] FAT-fs (md3): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[ 29.027295] XFS (dm-0): Mounting V5 Filesystem
[ 29.056344] XFS (dm-0): Starting recovery (logdev: internal)
[ 29.065516] XFS (dm-0): Ending recovery (logdev: internal)

Now the order I did the reboots:
- normal reboot from the cli (Initially did this)
- Poweroff via LOM (Did this when the normal reboot did not fix the issue with the one port that was not coming up)
- Hard reset (Did this after the Poweroff made things worse ie. all the module ports not working), and this fixed everything.

Module part code is : CPAC-8-1/10F-D
SFP modules are 10GB-SR and 1GB copper SFPs

Re: 9800 interfaces links down but switchports up

genisis__ — Sun, 12 Oct 2025 10:28:10 GMT

There was more going on here the_rock, but your right normally this would be what I would do.

Re: 9800 interfaces links down but switchports up

the_rock — Sun, 12 Oct 2025 11:17:03 GMT

Having read you last response, definitely makes more sense now.

Andy

Re: 9800 interfaces links down but switchports up

genisis__ — Sun, 12 Oct 2025 11:31:27 GMT

Also its not obvious but to run hardware diagnostics on a 9000 appliance:
R81 and higher versions support running the Diagnostic Tool in the Expert mode. The applicable command is:
expert# diagMain (SK97251) Note: When exiting the diagMain tool it will reboot the node, so be aware of this!

+--------------------------------------+
| HW Diagnostics |
+--------------------------------------+
| |
| |
| The system will be rebooted now... |
| |
| |
| |
+--------------------------------------+

Use <Up> and <Down> arrow keys to navigate
<ENTER> to select the menu item, <q> to exit current menu
Broadcast message from admin@FW (Sun Oct 12 12:28:24 2025):

The system is going down for reboot NOW!

Message from syslogd@FW at Oct 12 12:28:29 ...
monitord[18996]: monitord got killed

and in order to save logs you do need to connect a USB to the unit which makes sense, but would be nice if you had the option to store on the appliance as well.

Re: 9800 interfaces links down but switchports up

the_rock — Sun, 12 Oct 2025 11:25:54 GMT

Had no idea, thats super useful, THANK YOU!