topic Re: SMB cluster XL detailed logs in Spark Firewall (SMB)

SMB cluster XL detailed logs

vladdar — Fri, 03 Nov 2023 13:20:04 GMT

Hello all,

I am having a hard time to find detailed logs on cluster XL events on a pair of Quantum Spark 1800 with R80.20.40 (992002691). I know the firmware is a bit older but I faced issues when trying to upgrade to R81.10 (basically I was unable to form a cluster after an upgrade).

There is a state change in output bellow and I can't find a reason what happened and what it means State change: ACTIVE(!) -> ACTIVE

How can I find more details on this platform ? There is nothing interesting in /var/log/messages and

Gaia clish <show cluster member> doesn't work on Spark.

Thank You.

[Expert@gw1]#cphaprob stat

Cluster Mode: High Availability (Active Up) with IGMP Membership

ID Unique Address Assigned Load State

1 (local) 10.231.149.1 100% ACTIVE
2 10.231.149.2 0% STANDBY

Active PNOTEs: None

Last member state change event:
Event Code: CLUS-114904
State change: ACTIVE(!) -> ACTIVE
Reason for state change: Reason for ACTIVE! alert has been resolved
Event time: Fri Nov 3 13:09:43 2023

Last cluster failover event:
Transition to new ACTIVE: Member 2 -> Member 1
Reason: Incorrect configuration - Local cluster member has fewer cluster interfaces configured compared to other cluster member(s)
Event time: Fri Nov 3 10:59:24 2023

Cluster failover count:
Failover counter: 20
Time of counter reset: Fri Nov 3 13:09:43 2023 (reboot)

[Expert@gw1]# cphaprob show_failover

Cluster failover count:
Failover counter: 20
Time of counter reset: Fri Nov 3 13:09:43 2023 (reboot)

Cluster failover history (last 20 failovers since reboot/reset on Fri Nov 3 13:09:43 2023):

No. Time: Transition: CPU: Reason:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 Fri Nov 3 10:59:24 2023 Member 2 -> Member 1 00 Incorrect configuration - Local cluster member has fewer cluster interfaces configured compared to other cluster member(s)
2 Thu Oct 5 20:54:38 2023 Member 1 -> Member 2 00 Incorrect configuration - Local cluster member has fewer cluster interfaces configured compared to other cluster member(s)
3 Sun Sep 17 12:49:25 2023 Member 2 -> Member 1 00 Interface LANBOND0 is down (Cluster Control Protocol packets are not received)
4 Sun Sep 17 11:44:53 2023 Member 1 -> Member 2 00 Incorrect configuration - Local cluster member has fewer cluster interfaces configured compared to other cluster member(s)

Re: SMB cluster XL detailed logs

Chris_Atkinson — Fri, 03 Nov 2023 14:20:08 GMT

Which process did you follow in attempting the upgrade?

You can try checking the following log(s) for additional information to start, but you may need to consult further with TAC.

/var/log/messages*

$FWDIR/log/sfwd.elg*

Re: SMB cluster XL detailed logs

vladdar — Fri, 03 Nov 2023 14:53:20 GMT

Thank you, looks like I have found something in $FWDIR/log/sfwd.elg*

nothing in /var/log/messages*

Altough, I do not fully understand it. Something happened in that vlan, could it affect the whole cluster (other vlans) ? Unfortunately, logs on primary member have rotated so I don't have them.

[sfwd 9105 4154539136]@gw2[3 Nov 13:03:33] Cluster member status changed from STANDBY to DOWN.
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Detected route change:
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Op=route-del
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Interface=LANBOND0.3
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Dst=10.0.0.96
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Mask=255.255.255.248
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Gateway=10.3.3.2
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Detected route change:
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Op=route-del
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Interface=LANBOND0.3
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Dst=10.0.0.104
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Mask=255.255.255.248
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Gateway=10.3.3.2
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Detected route change:
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Op=route-del
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Interface=LANBOND0.3
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Dst=10.0.0.248
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Mask=255.255.255.248
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Gateway=10.3.3.2
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Detected route change:
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Op=route-del
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Interface=LANBOND0.3
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Dst=10.0.6.0
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Mask=255.255.255.0
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Gateway=10.3.3.2
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Detected route change:
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Op=route-del
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Interface=LANBOND0.3
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Dst=10.2.0.0
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Mask=255.255.0.0
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Gateway=10.3.3.2
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:34] Cluster member status changed from DOWN to STANDBY.
[sfwd 9105 4154539136]@gw2[3 Nov 13:03:37] Requesting settings synchronization from cluster peer.

[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Detected route change:
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Op=route-add
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Interface=LANBOND0.3
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Dst=10.0.0.96
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Mask=255.255.255.248
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Gateway=10.3.3.2
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Detected route change:
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Op=route-add
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Interface=LANBOND0.3
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Dst=10.0.0.104
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Mask=255.255.255.248
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Gateway=10.3.3.2
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Detected route change:
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Op=route-add
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Interface=LANBOND0.3
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Dst=10.0.0.248
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Mask=255.255.255.248
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Gateway=10.3.3.2
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Detected route change:
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Op=route-add
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Interface=LANBOND0.3
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Dst=10.0.6.0
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Mask=255.255.255.0
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Gateway=10.3.3.2
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Detected route change:
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Op=route-add
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Interface=LANBOND0.3
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Dst=10.2.0.0
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Mask=255.255.0.0
[sfwd 9105 4154539136]@gw2[3 Nov 13:09:44] Gateway=10.3.3.2

Re: SMB cluster XL detailed logs

PhoneBoy — Fri, 03 Nov 2023 19:20:20 GMT

Any changes to the interfaces/VLANs is likely to impact cluster state.