Incorrect configuration - Local cluster member has...

ipolovokhin · ‎2022-12-05

Good afternoon.
After a long search on the forums decided to write here.

I have a problem with the cluster:

Cluster Mode: High Availability (Active Up) with IGMP Membership

ID Unique Address Assigned Load State Name

1 (local) 172.29.0.185 0% DOWN BV09-200-WAN-FW-003
2 172.29.0.186 100% ACTIVE BV09-200-WAN-FW-004

Active PNOTEs: IAC

Last member state change event:
Event Code: CLUS-110805
State change: ACTIVE(!) -> DOWN
Reason for state change: Incorrect configuration - Local cluster member has fewer cluster interfaces configured compared to other cluster member(s)
Event time: Fri Feb 18 13:04:12 2022

Last cluster failover event:
Transition to new ACTIVE: Member 1 -> Member 2
Reason: Incorrect configuration - Local cluster member has fewer cluster interfaces configured compared to other cluster member(s)
Event time: Fri Feb 18 13:04:39 2022

Cluster failover count:
Failover counter: 3
Time of counter reset: Mon Apr 12 11:37:18 2021 (reboot)

For clarity, the output of the same command from the second cluster member:

Cluster Mode: High Availability (Active Up) with IGMP Membership

ID Unique Address Assigned Load State Name

1 172.29.0.185 0% DOWN BV09-200-WAN-FW-003
2 (local) 172.29.0.186 100% ACTIVE BV09-200-WAN-FW-004

Active PNOTEs: None

Last member state change event:
Event Code: CLUS-114904
State change: ACTIVE(!) -> ACTIVE
Reason for state change: Reason for ACTIVE! alert has been resolved
Event time: Fri Feb 18 13:14:04 2022

Last cluster failover event:
Transition to new ACTIVE: Member 1 -> Member 2
Reason: Incorrect configuration - Local cluster member has fewer cluster interfaces configured compared to other cluster member(s)
Event time: Fri Feb 18 13:04:39 2022

Cluster failover count:
Failover counter: 3
Time of counter reset: Mon Apr 12 11:37:18 2021 (reboot)

Seeing this, I decided to look at the list of interfaces. And found some inconsistencies.

Interfaces on the problematic cluster member:

[Expert@200-BV09-WAN-FW-003:0]# cphaprob -a if

CCP mode: Manual (Unicast)
Required interfaces: 3
Required secured interfaces: 1

Interface Name: Status:

eth5 (S) UP
Mgmt Non-Monitored
eth1-01 UP
eth1-02 UP

S - sync, LM - link monitor, HA/LS - bond type

Virtual cluster interfaces: 12

eth1-01 172.27.0.65
eth1-02 172.28.0.65
bond10.300 172.18.10.1
bond10.1041 10.10.41.1
bond10.1042 10.10.42.1
bond10.1031 10.10.31.1
bond10.1032 10.10.32.1
bond10.1033 10.10.33.1
bond10.1047 10.10.47.1
bond10.1048 10.10.48.1
bond10.1044 10.10.44.1
bond10.1045 10.10.45.1

Interfaces on an Active Cluster Member:

CCP mode: Manual (Unicast)
Required interfaces: 5
Required secured interfaces: 1

Interface Name: Status:

eth5 (S) UP
Mgmt Non-Monitored
eth1-01 UP
eth1-02 UP
bond10.300 (LS) UP
bond10.1048 (LS) UP

S - sync, LM - link monitor, HA/LS - bond type

Virtual cluster interfaces: 12

eth1-01 172.27.0.65
eth1-02 172.28.0.65
bond10.300 172.18.10.1
bond10.1041 10.10.41.1
bond10.1042 10.10.42.1
bond10.1031 10.10.31.1
bond10.1032 10.10.32.1
bond10.1033 10.10.33.1
bond10.1047 10.10.47.1
bond10.1048 10.10.48.1
bond10.1044 10.10.44.1
bond10.1045 10.10.45.1

For some reason, there are no bond interfaces on the problematic cluster member in load sharing mode.

The cluster is connected by these bond interfaces to one stack of switches. But to different aggregation groups. This is not the only location. At other locations, the topology is identical and there are no problems with the cluster.

I will be very grateful for your help.
Have a good day!

PhoneBoy · ‎2022-12-05

I’m not clear why you’re showing one set of cluster members with HA using IGMP and another using Unicast.
Regardless, the number of interfaces (and their state) must match on both cluster members and bond10 appears to be problematic.
What precise troubleshooting have you done on this interface?
What does ifconfig say, for instance?

ipolovokhin · ‎2022-12-05

Good afternoon.

I didn't quite understand what you meant. Shown at the top are the outputs of the cphaprob stat command.

Next, I gave the output of the sphaprob -a if command to show that in the case of one Firewall there are 2 LS interfaces and in the other case there are not.

Ifconfig command output from firewall in DOWN state:

[Expert@200-BV09-WAN-FW-003:0]# ifconfig
Mgmt Link encap:Ethernet HWaddr 00:1C:7F:8B:D0:79
inet addr:172.20.103.162 Bcast:172.20.103.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:66468071 errors:0 dropped:1 overruns:0 frame:0
TX packets:133504235 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:10840530384 (10.0 GiB) TX bytes:68632684371 (63.9 GiB)

Mgmt:1 Link encap:Ethernet HWaddr 00:1C:7F:8B:D0:79
inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

bond10 Link encap:Ethernet HWaddr 00:1C:7F:6C:90:55
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:936579814 errors:0 dropped:0 overruns:0 frame:0
TX packets:2872096493 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:191541144420 (178.3 GiB) TX bytes:166658055171 (155.2 GiB)

bond10.300 Link encap:Ethernet HWaddr 00:1C:7F:6C:90:55
inet addr:172.18.10.2 Bcast:172.18.10.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:557956754 errors:0 dropped:0 overruns:0 frame:0
TX packets:1084985824 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:153402974007 (142.8 GiB) TX bytes:79938662070 (74.4 GiB)

bond10.1031 Link encap:Ethernet HWaddr 00:1C:7F:6C:90:55
inet addr:10.10.31.2 Bcast:10.10.31.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12794127 errors:0 dropped:0 overruns:0 frame:0
TX packets:4859123 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:824757716 (786.5 MiB) TX bytes:327728128 (312.5 MiB)

bond10.1032 Link encap:Ethernet HWaddr 00:1C:7F:6C:90:55
inet addr:10.10.32.2 Bcast:10.10.32.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:417590 errors:0 dropped:0 overruns:0 frame:0
TX packets:215874 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:17538780 (16.7 MiB) TX bytes:9067048 (8.6 MiB)

bond10.1033 Link encap:Ethernet HWaddr 00:1C:7F:6C:90:55
inet addr:10.10.33.2 Bcast:10.10.33.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:417596 errors:0 dropped:0 overruns:0 frame:0
TX packets:215874 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:17539032 (16.7 MiB) TX bytes:9067048 (8.6 MiB)

bond10.1041 Link encap:Ethernet HWaddr 00:1C:7F:6C:90:55
inet addr:10.10.41.2 Bcast:10.10.41.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:147419485 errors:0 dropped:53 overruns:0 frame:0
TX packets:132429508 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:10207559105 (9.5 GiB) TX bytes:7284030775 (6.7 GiB)

bond10.1042 Link encap:Ethernet HWaddr 00:1C:7F:6C:90:55
inet addr:10.10.42.2 Bcast:10.10.42.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:106994760 errors:0 dropped:43 overruns:0 frame:0
TX packets:99419634 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7159634515 (6.6 GiB) TX bytes:5391734778 (5.0 GiB)

bond10.1044 Link encap:Ethernet HWaddr 00:1C:7F:6C:90:55
inet addr:10.10.44.2 Bcast:10.10.44.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:5625381 errors:0 dropped:615 overruns:0 frame:0
TX packets:4960344 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:347907927 (331.7 MiB) TX bytes:273938961 (261.2 MiB)

bond10.1045 Link encap:Ethernet HWaddr 00:1C:7F:6C:90:55
inet addr:10.10.45.2 Bcast:10.10.45.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:429331 errors:0 dropped:156 overruns:0 frame:0
TX packets:180647 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:21353556 (20.3 MiB) TX bytes:7587514 (7.2 MiB)

bond10.1047 Link encap:Ethernet HWaddr 00:1C:7F:6C:90:55
inet addr:10.10.47.2 Bcast:10.10.47.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:417590 errors:0 dropped:0 overruns:0 frame:0
TX packets:215874 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:17538780 (16.7 MiB) TX bytes:9067048 (8.6 MiB)

bond10.1048 Link encap:Ethernet HWaddr 00:1C:7F:6C:90:55
inet addr:10.10.48.2 Bcast:10.10.48.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:100640538 errors:0 dropped:0 overruns:0 frame:0
TX packets:1441407523 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6030890658 (5.6 GiB) TX bytes:60539275230 (56.3 GiB)

eth1-01 Link encap:Ethernet HWaddr 00:1C:7F:6C:90:53
inet addr:172.27.0.66 Bcast:172.27.0.71 Mask:255.255.255.248
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:509827918 errors:0 dropped:0 overruns:0 frame:0
TX packets:360284645 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:66868162171 (62.2 GiB) TX bytes:63842485487 (59.4 GiB)

eth1-02 Link encap:Ethernet HWaddr 00:1C:7F:6C:90:54
inet addr:172.28.0.66 Bcast:172.28.0.71 Mask:255.255.255.248
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:132118147 errors:0 dropped:0 overruns:0 frame:0
TX packets:476044823 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:11972860897 (11.1 GiB) TX bytes:97329396632 (90.6 GiB)

eth1-03 Link encap:Ethernet HWaddr 00:1C:7F:6C:90:55
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:931454287 errors:0 dropped:0 overruns:0 frame:0
TX packets:2820020821 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:191081126150 (177.9 GiB) TX bytes:160204003503 (149.2 GiB)

eth1-04 Link encap:Ethernet HWaddr 00:1C:7F:6C:90:55
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:5123424 errors:0 dropped:0 overruns:0 frame:0
TX packets:52068526 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:459876474 (438.5 MiB) TX bytes:6453165564 (6.0 GiB)

eth5 Link encap:Ethernet HWaddr 00:1C:7F:8B:D0:78
inet addr:172.29.0.185 Bcast:172.29.0.187 Mask:255.255.255.252
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1477470924 errors:0 dropped:0 overruns:0 frame:0
TX packets:1476028222 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:147159815368 (137.0 GiB) TX bytes:126798951263 (118.0 GiB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING PROMISC ALLMULTI MTU:65536 Metric:1
RX packets:356327509 errors:0 dropped:0 overruns:0 frame:0
TX packets:356327509 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:76797021997 (71.5 GiB) TX bytes:76797021997 (71.5 GiB)

Ifconfig command output from firewall in DOWN state:

[Expert@200-BV09-WAN-FW-004:0]# ifconfig
Mgmt Link encap:Ethernet HWaddr 00:1C:7F:8B:C7:43
inet addr:172.20.103.163 Bcast:172.20.103.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:71702730 errors:0 dropped:1 overruns:0 frame:0
TX packets:40636746 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:11168844275 (10.4 GiB) TX bytes:38429831816 (35.7 GiB)

Mgmt:1 Link encap:Ethernet HWaddr 00:1C:7F:8B:C7:43
inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

bond10 Link encap:Ethernet HWaddr 00:1C:7F:6C:93:1D
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:1236539414 errors:0 dropped:0 overruns:0 frame:0
TX packets:1749755191 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:302729612955 (281.9 GiB) TX bytes:173628661293 (161.7 GiB)

bond10.300 Link encap:Ethernet HWaddr 00:1C:7F:6C:93:1D
inet addr:172.18.10.3 Bcast:172.18.10.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:646285201 errors:0 dropped:0 overruns:0 frame:0
TX packets:1014145496 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:245039320577 (228.2 GiB) TX bytes:124843763325 (116.2 GiB)

bond10.1031 Link encap:Ethernet HWaddr 00:1C:7F:6C:93:1D
inet addr:10.10.31.3 Bcast:10.10.31.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:25357673 errors:0 dropped:0 overruns:0 frame:0
TX packets:16539743 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1735422445 (1.6 GiB) TX bytes:1106337297 (1.0 GiB)

bond10.1032 Link encap:Ethernet HWaddr 00:1C:7F:6C:93:1D
inet addr:10.10.32.3 Bcast:10.10.32.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1 errors:0 dropped:0 overruns:0 frame:0
TX packets:650979 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:42 (42.0 b) TX bytes:27341262 (26.0 MiB)

bond10.1033 Link encap:Ethernet HWaddr 00:1C:7F:6C:93:1D
inet addr:10.10.33.3 Bcast:10.10.33.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1 errors:0 dropped:0 overruns:0 frame:0
TX packets:650985 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:42 (42.0 b) TX bytes:27341514 (26.0 MiB)

bond10.1041 Link encap:Ethernet HWaddr 00:1C:7F:6C:93:1D
inet addr:10.10.41.3 Bcast:10.10.41.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:242150665 errors:0 dropped:53 overruns:0 frame:0
TX packets:220279126 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:16419085419 (15.2 GiB) TX bytes:12147944929 (11.3 GiB)

bond10.1042 Link encap:Ethernet HWaddr 00:1C:7F:6C:93:1D
inet addr:10.10.42.3 Bcast:10.10.42.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:238273304 errors:0 dropped:56 overruns:0 frame:0
TX packets:213709555 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:15767209379 (14.6 GiB) TX bytes:11714066031 (10.9 GiB)

bond10.1044 Link encap:Ethernet HWaddr 00:1C:7F:6C:93:1D
inet addr:10.10.44.3 Bcast:10.10.44.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:30894725 errors:0 dropped:139 overruns:0 frame:0
TX packets:23043171 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3970953639 (3.6 GiB) TX bytes:1768575833 (1.6 GiB)

bond10.1045 Link encap:Ethernet HWaddr 00:1C:7F:6C:93:1D
inet addr:10.10.45.3 Bcast:10.10.45.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1 errors:0 dropped:0 overruns:0 frame:0
TX packets:650979 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:42 (42.0 b) TX bytes:27341262 (26.0 MiB)

bond10.1047 Link encap:Ethernet HWaddr 00:1C:7F:6C:93:1D
inet addr:10.10.47.3 Bcast:10.10.47.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1 errors:0 dropped:0 overruns:0 frame:0
TX packets:650979 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:42 (42.0 b) TX bytes:27341262 (26.0 MiB)

bond10.1048 Link encap:Ethernet HWaddr 00:1C:7F:6C:93:1D
inet addr:10.10.48.3 Bcast:10.10.48.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:50110193 errors:0 dropped:0 overruns:0 frame:0
TX packets:1491601957 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2104628142 (1.9 GiB) TX bytes:65153350654 (60.6 GiB)

eth1-01 Link encap:Ethernet HWaddr 00:1C:7F:6C:93:1B
inet addr:172.27.0.67 Bcast:172.27.0.71 Mask:255.255.255.248
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:620639591 errors:0 dropped:0 overruns:0 frame:0
TX packets:482753707 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:98368857374 (91.6 GiB) TX bytes:156119149825 (145.3 GiB)

eth1-02 Link encap:Ethernet HWaddr 00:1C:7F:6C:93:1C
inet addr:172.28.0.67 Bcast:172.28.0.71 Mask:255.255.255.248
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:300050082 errors:0 dropped:0 overruns:0 frame:0
TX packets:862659259 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:41732304216 (38.8 GiB) TX bytes:159919844589 (148.9 GiB)

eth1-03 Link encap:Ethernet HWaddr 00:1C:7F:6C:93:1D
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:1057700720 errors:0 dropped:0 overruns:0 frame:0
TX packets:1639378219 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:267160282388 (248.8 GiB) TX bytes:143252084231 (133.4 GiB)

eth1-04 Link encap:Ethernet HWaddr 00:1C:7F:6C:93:1D
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:178837538 errors:0 dropped:0 overruns:0 frame:0
TX packets:110367323 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:35569240215 (33.1 GiB) TX bytes:30375380586 (28.2 GiB)

eth5 Link encap:Ethernet HWaddr 00:1C:7F:8B:C7:42
inet addr:172.29.0.186 Bcast:172.29.0.187 Mask:255.255.255.252
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1476019633 errors:0 dropped:0 overruns:0 frame:0
TX packets:1477462347 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:126836717777 (118.1 GiB) TX bytes:147120230445 (137.0 GiB)

lo Link encap:Local Loopback Media:unknown(auto)
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING NOARP ALLMULTI MULTICAST MTU:65536 Metric:1
RX packets:353745679 errors:0 dropped:0 overruns:0 frame:0
TX packets:353745679 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:76688628168 (71.4 GiB) TX bytes:76688628168 (71.4 GiB)

Yes, I know that the number of interfaces and their states must match. But I don't understand how to do it. I see that on one of the firewalls there are 5 interfaces and on the other there are three. I want to understand the reason for their appearance in order to think of a solution.

Perhaps this is a stupid question. But I didn't find an answer to it on this forum. Perhaps the answer to it will be useful to other people.

Thank you for your interest in my question.

ipolovokhin · ‎2022-12-05

No action has been taken so far. I just want to understand what the problem is at this stage.
ifconfig output looks fine (number of interfaces match and everything is UP). I wanted to attach the conclusions of the commands here, but did not miss the message.

Yes, you are right, the cluster is in High Availability (Active Up) with IGMP Membership mode. What in that case should be CCP mode?

PhoneBoy · ‎2022-12-05

For a cluster member to be up, the relevant interface must be up AND it must see traffic from the other member on that interface.
For Trunk interfaces, we only monitor the upper and lower-most VLAN (by VLAN ID).
Which means…if I’m looking at troubleshooting, I’d start by doing a tcpdump on the relevant VLANs to see if you can see the CCP packets (UDP port 8116).

I forgot we switched CCP mode to Unicast in R80.40 and above…therefore my comments about it aren’t entirely relevant.
In any case, you can start here for general troubleshooting of ClusterXL issues: https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solut...

Matlu · ‎2023-12-18

Hello,

I have the same behaviour with my ClusterXL HA in version R81.10

The topology in the SmartConsole

Cluster object looks "fine".

The only different thing I see is the output of the "cphaprob -a if", because the problematic member shows the option "Required interfaces: 5" and the now working member shows "Required interfaces: 6"

Is it necessary to restart the process with "cphastop;cphastart"?

Member DOWN

CCP mode: Manual (Unicast)
Required interfaces: 5
Required secured interfaces: 1

Interface Name: Status:

eth2 UP
Sync (S) UP
Mgmt UP
eth1.19 UP
eth1.2 UP

S - sync, HA/LS - bond type, LM - link monitor, P - probing

Virtual cluster interfaces: 9

eth2 192.168.60.1
Mgmt 192.168.128.27
eth1.8 192.168.8.254
eth1.6 192.168.7.254
eth1.19 192.168.19.254
eth1.4 192.168.1.254
eth1.3 192.168.2.254
eth1.2 192.168.5.254
eth1.15 192.168.15.254

--------------------------------------------------------------------------------------------------------

Active Member

CCP mode: Manual (Unicast)
Required interfaces: 6
Required secured interfaces: 1

Interface Name: Status:

eth2 UP
Sync (S) UP
Mgmt UP
eth1.19 UP
eth1.8 (P) UP
eth1.2 UP

S - sync, HA/LS - bond type, LM - link monitor, P - probing

Virtual cluster interfaces: 9

eth2 192.168.60.1
Mgmt 192.168.128.27
eth1.19 192.168.19.254
eth1.8 192.168.8.254
eth1.6 192.168.7.254
eth1.2 192.168.5.254
eth1.3 192.168.2.254
eth1.15 192.168.15.254
eth1.4 192.168.1.254

Cheers.

the_rock · ‎2023-12-18

Buenos diaz bro,

How are you? Its clear to me from that output issue is with eth1.8, as its not present on the other fw. Please verify its set as cluster in the topology.

Andy

Matlu · ‎2023-12-18

Hey, Andy

It looks good.
It's strange 😕

the_rock · ‎2023-12-18

Its a bit tricky with cphastop; cphastart as you may have to do it on BOTH firewalls, so probably best after hours. You can try only on problematic one, but no guarantee that would fix the issue.

Andy

Matlu · ‎2023-12-18

If you apply the "cphastop;cphastart" on a working window, would you start applying these commands on the "Active" member?

Or can I do it, on the "Downed Member"?

the_rock · ‎2023-12-18

Thats what I said, do it on downed member, see what happens. I cant tell for sure if that will work, but you can try. If it does not work, you may have to do it on working member too. Had same scenario with TAC on the phone 4 times in the past.

Andy

the_rock · ‎2022-12-05

Just some thoughts...

-can you confirm topology looks correct for problematic interface?

-also, any way you can run cphastop and cphastart on problematic member showing down?

-can you send output of cphaprob syncstat?

Andy

Lender · ‎2023-08-10

What does cphaprob tablestat give you?
Might be a good way of comparing what's what with respect to HA.

Are you a member of CheckMates?

Incorrect configuration - Local cluster member has fewer cluster interfaces configured compared to