topic Re: cluster fail in General Topics

cluster fail

ChinChang — Mon, 09 Dec 2024 12:07:16 GMT

Dears,
I'm learning cluster, but I met some trouble.
Please refer my topology & lab steps, and my SMS and 2 gateways version are R82.
I want to add cp101 & cp102 into my cluster, when I finish cluster operate and press publish, it failed.
Expect get help, tks.

Re: cluster fail

PhoneBoy — Mon, 09 Dec 2024 17:50:44 GMT

What was the precise error received when you attempted to publish?

Re: cluster fail

the_rock — Mon, 09 Dec 2024 23:54:06 GMT

If you send us the error you get, it would help, for sure.

Andy

Re: cluster fail

Lesley — Tue, 10 Dec 2024 06:24:45 GMT

Looks like you have not defined eth2 as sync interface is this correct? Clusters needs to have a sync interface between 2 firewalls.

Re: cluster fail

ChinChang — Wed, 11 Dec 2024 05:27:25 GMT

I try change eth2 to sync interface, still same result.

Re: cluster fail

ChinChang — Wed, 11 Dec 2024 05:28:23 GMT

after publish by smartconsole. it show alert about clusterXL as screenshot, i think my lab should install it, and then i did also.

Re: cluster fail

ChinChang — Wed, 11 Dec 2024 05:28:52 GMT

after publish by smartconsole. it show alert about clusterXL as screenshot, i think my lab should install it, and then i did also.

Re: cluster fail

PhoneBoy — Wed, 11 Dec 2024 13:38:54 GMT

I'd start with disabling and re-enabling ClusterXL in cpconfig: https://support.checkpoint.com/results/sk/sk88360

Re: cluster fail

Martijn — Thu, 12 Dec 2024 13:19:29 GMT

Hi,

When following the First Time Wizard on the appliances, did you select 'part of a cluster'?
As Dameon says, is cluster membership enabled when you check 'cpconfig'.

Please send us the output of 'cpconfig' from the gateways.
Can you also share (in expert mode) the output of 'cphaprob stat' and 'cphaprob -l list'?

Looks like clustering is not enabled on the members.

Regards,
Martijn

Re: cluster fail

the_rock — Thu, 12 Dec 2024 13:31:44 GMT

Can you please send a screenshot of publish failure you get?

Andy

Re: cluster fail

ChinChang — Sat, 14 Dec 2024 09:07:41 GMT

thanks your support, i tried it, disable and then re-enable, still failed.

Re: cluster fail

ChinChang — Sat, 14 Dec 2024 09:08:04 GMT

it had not publish fail,
in fact, publish success, but after publish like screenshot, status are red alert.

Re: cluster fail

ChinChang — Sat, 14 Dec 2024 09:13:43 GMT

did you select 'part of a cluster'?
>>yes, in cp101 & cp102.

cp101> cpconfig
This program will let you re-configure
your Check Point products configuration.

Configuration Options:
----------------------
(1) Licenses and contracts
(2) SNMP Extension
(3) PKCS#11 Token
(4) Random Pool
(5) Secure Internal Communication
(6) Disable cluster membership for this gateway
(7) Enable Check Point Per Virtual System State
(8) Disable Check Point ClusterXL for Bridge Active/Standby
(9) Check Point CoreXL
(10) Automatic start of Check Point Products

(11) Exit

Enter your choice (1-11) :11

Thank You...
cp101> expert
Enter expert password:

Warning! All configurations should be done through clish
You are in expert mode now.

[Expert@cp101:0]# cphaprob stat

HA module not started.

Cluster policy installation failed on gateway (Error code: 204).
(see sk125152).[Expert@cp101:0]#
[Expert@cp101:0]# cphaprob -l list

Built-in Devices:

Device Name: Interface Active Check

Device Name: Recovery Delay

Device Name: CoreXL Configuration

Registered Devices:

Device Name: Fullsync
Registration number: 0
Timeout: none
Additional description: cpstop

Device Name: Policy
Registration number: 1
Timeout: none

Device Name: routed
Registration number: 2
Timeout: none
Additional description: Cluster Start

Device Name: cxld
Registration number: 3
Timeout: 30 sec

Device Name: HD
Registration number: 4
Timeout: none

Device Name: fwd
Registration number: 5
Timeout: 30 sec

[Expert@cp101:0]#

===============================================================================

cp102> cpconfig
This program will let you re-configure
your Check Point products configuration.

(11) Exit

Enter your choice (1-11) :11

Thank You...
cp102> expert
Enter expert password:

Warning! All configurations should be done through clish
You are in expert mode now.

[Expert@cp102:0]# cphaprob stat

HA module not started.

Cluster policy should be installed - please run cphastart
[Expert@cp102:0]# cphaprob -l list

Built-in Devices:

Device Name: Interface Active Check

Device Name: Recovery Delay

Device Name: CoreXL Configuration

Registered Devices:

Device Name: Fullsync
Registration number: 0
Timeout: none
Additional description: cpstop

Device Name: Policy
Registration number: 1
Timeout: none

Device Name: routed
Registration number: 2
Timeout: none
Additional description: Cluster Start

Device Name: cxld
Registration number: 3
Timeout: 30 sec

Device Name: HD
Registration number: 4
Timeout: none

Device Name: fwd
Registration number: 5
Timeout: 30 sec

[Expert@cp102:0]#

Re: cluster fail

the_rock — Sat, 14 Dec 2024 13:26:10 GMT

Can you send below commands from expert mode from BOTH members?

Andy

cphaprob roles

cphaprob state

cphaprob -a if

cphaprob -i list

cphaprob -l list

cphaprob syncstat

I would say if you cant figure it out, try cphastop; cphastart on both members, if still nothing, run cpconfig, disable cluster membership, reboot, re-enable, reboot (do on both). If still no luck, I think remote session might be needed to verify everything.

Andy

Re: cluster fail

ChinChang — Sun, 15 Dec 2024 18:23:32 GMT

did you select 'part of a cluster'?
>>yes, i did. in cp1 & cp2.

cp1> cpconfig
This program will let you re-configure
your Check Point products configuration.

Configuration Options:
----------------------
(1) Licenses and contracts
(2) SNMP Extension
(3) PKCS#11 Token
(4) Random Pool
(5) Secure Internal Communication
(6) Disable cluster membership for this gateway
(7) Enable Check Point Per Virtual System State
(8) Enable Check Point ClusterXL for Bridge Active/Standby
(9) Check Point CoreXL
(10) Automatic start of Check Point Products

(11) Exit

Enter your choice (1-11) :11

Thank You...
cp1> cphaprob stat

HA module not started.

Cluster policy should be installed - please run cphastart
cp1>
cp1> cphaprob -l list

Built-in Devices:

Device Name: Interface Active Check

Device Name: Recovery Delay

Device Name: CoreXL Configuration

Registered Devices:

Device Name: Fullsync
Registration number: 0
Timeout: none
Additional description: Boot

Device Name: Policy
Registration number: 1
Timeout: none

Device Name: routed
Registration number: 2
Timeout: none
Additional description: Cluster Start

Device Name: cxld
Registration number: 3
Timeout: 30 sec

Device Name: HD
Registration number: 4
Timeout: none

Device Name: fwd
Registration number: 5
Timeout: 30 sec

cp1>
=====================================
cp2> cpconfig
This program will let you re-configure
your Check Point products configuration.

Configuration Options:
----------------------
(1) Licenses and contracts
(2) SNMP Extension
(3) PKCS#11 Token
(4) Random Pool
(5) Secure Internal Communication
(6) Disable cluster membership for this gateway
(7) Enable Check Point Per Virtual System State
(8) Enable Check Point ClusterXL for Bridge Active/Standby
(9) Check Point CoreXL
(10) Automatic start of Check Point Products

(11) Exit

Enter your choice (1-11) :11

Thank You...
cp2> clear
cp2> cpconfig
This program will let you re-configure
your Check Point products configuration.

Configuration Options:
----------------------
(1) Licenses and contracts
(2) SNMP Extension
(3) PKCS#11 Token
(4) Random Pool
(5) Secure Internal Communication
(6) Disable cluster membership for this gateway
(7) Enable Check Point Per Virtual System State
(8) Enable Check Point ClusterXL for Bridge Active/Standby
(9) Check Point CoreXL
(10) Automatic start of Check Point Products

(11) Exit

Enter your choice (1-11) :11

Thank You...
cp2> cphaprob stat

HA module not started.

Cluster policy should be installed - please run cphastart
cp2> cphaprob -l list

Built-in Devices:

Device Name: Interface Active Check

Device Name: Recovery Delay

Device Name: CoreXL Configuration

Registered Devices:

Device Name: Fullsync
Registration number: 0
Timeout: none
Additional description: Boot

Device Name: Policy
Registration number: 1
Timeout: none

Device Name: routed
Registration number: 2
Timeout: none
Additional description: Cluster Start

Device Name: cxld
Registration number: 3
Timeout: 30 sec

Device Name: HD
Registration number: 4
Timeout: none

Device Name: fwd
Registration number: 5
Timeout: 30 sec

cp2>

Re: cluster fail

the_rock — Sun, 15 Dec 2024 18:33:58 GMT

Did you run cphastart?

Andy

Re: cluster fail

ChinChang — Sun, 15 Dec 2024 18:44:13 GMT

i try it, cphastop and then cphastart, i got same result.
and below is follow your guide.

cp1> expert
Enter expert password:

Warning! All configurations should be done through clish
You are in expert mode now.

[Expert@cp1:0]# cphaprob roles

ID Role

[Expert@cp1:0]#
[Expert@cp1:0]# cphaprob state

HA module not started.

Cluster policy should be installed - please run cphastart
[Expert@cp1:0]# cphaprob -a if

HA module not started.

Warning: Sync will not function since there aren't any sync(secured) interfaces

[Expert@cp1:0]# cphaprob -i list

Built-in Devices:

Device Name: Interface Active Check

Device Name: HA Initialization

Device Name: Recovery Delay

Device Name: CoreXL Configuration

Registered Devices:

Device Name: Fullsync
Registration number: 0
Timeout: none
Additional description: Boot

Device Name: routed
Registration number: 2
Timeout: none
Additional description: Cluster Start

[Expert@cp1:0]# cphaprob -l list

Built-in Devices:

Device Name: Interface Active Check

Device Name: Recovery Delay

Device Name: CoreXL Configuration

Registered Devices:

Device Name: Fullsync
Registration number: 0
Timeout: none
Additional description: Boot

Device Name: Policy
Registration number: 1
Timeout: none

Device Name: routed
Registration number: 2
Timeout: none
Additional description: Cluster Start

Device Name: cxld
Registration number: 3
Timeout: 30 sec

Device Name: HD
Registration number: 4
Timeout: none

Device Name: fwd
Registration number: 5
Timeout: 30 sec

[Expert@cp1:0]# cphaprob syncstat

Delta Sync Statistics

Sync status: Off - Sync interface down.

Drops:
Lost updates................................. 0
Lost bulk update events...................... 0
Oversized updates not sent................... 0

Sync at risk:
Sent reject notifications.................... 0
Received reject notifications................ 0

Sent messages:
Total generated sync messages................ 0
Sent retransmission requests................. 0
Sent retransmission updates.................. 0
Peak fragments per update.................... 1

Received messages:
Total received updates....................... 0
Received retransmission requests............. 0

Sync Interface:
Name.........................................
Link speed...................................
Rate......................................... 0 [Bps]
Peak rate.................................... 0 [Bps]
Link usage................................... 0%
Total........................................ 0 [B]

Queue sizes (num of updates):
Sending queue size........................... 512
Receiving queue size......................... 256
Fragments queue size......................... 50

Timers:
Delta Sync interval (ms)..................... 100

Reset on Mon Dec 16 10:12:39 2024 (triggered by fullsync).

[Expert@cp1:0]#
====================================================
cp2> expert
Enter expert password:

Warning! All configurations should be done through clish
You are in expert mode now.

[Expert@cp2:0]# cphaprob roles

ID Role

[Expert@cp2:0]#
[Expert@cp2:0]# cphaprob state

HA module not started.

Cluster policy should be installed - please run cphastart
[Expert@cp2:0]#
[Expert@cp2:0]# cphaprob -a if

HA module not started.

Warning: Sync will not function since there aren't any sync(secured) interfaces

[Expert@cp2:0]#
[Expert@cp2:0]# cphaprob -i list

Built-in Devices:

Device Name: Interface Active Check

Device Name: HA Initialization

Device Name: Recovery Delay

Device Name: CoreXL Configuration

Registered Devices:

Device Name: Fullsync
Registration number: 0
Timeout: none
Additional description: Boot

Device Name: routed
Registration number: 2
Timeout: none
Additional description: Cluster Start

[Expert@cp2:0]#
[Expert@cp2:0]# cphaprob -l list

Built-in Devices:

Device Name: Interface Active Check

Device Name: Recovery Delay

Device Name: CoreXL Configuration

Registered Devices:

Device Name: Fullsync
Registration number: 0
Timeout: none
Additional description: Boot

Device Name: Policy
Registration number: 1
Timeout: none

Device Name: routed
Registration number: 2
Timeout: none
Additional description: Cluster Start

Device Name: cxld
Registration number: 3
Timeout: 30 sec

Device Name: HD
Registration number: 4
Timeout: none

Device Name: fwd
Registration number: 5
Timeout: 30 sec

[Expert@cp2:0]#
[Expert@cp2:0]# cphaprob syncstat

Delta Sync Statistics

Sync status: Off - Sync interface down.

Drops:
Lost updates................................. 0
Lost bulk update events...................... 0
Oversized updates not sent................... 0

Sync at risk:
Sent reject notifications.................... 0
Received reject notifications................ 0

Received messages:
Total received updates....................... 0
Received retransmission requests............. 0

Queue sizes (num of updates):
Sending queue size........................... 512
Receiving queue size......................... 256
Fragments queue size......................... 50

Timers:
Delta Sync interval (ms)..................... 100

Reset on Mon Dec 16 10:12:54 2024 (triggered by fullsync).

[Expert@cp2:0]#

Re: cluster fail

the_rock — Sun, 15 Dec 2024 18:52:07 GMT

This is WHY it is NOT working. Can we do remote? Im free to assist if you are allowed to do zoom remote.

Andy

[Expert@cp1:0]# cphaprob -a if

HA module not started.

Warning: Sync will not function since there aren't any sync(secured) interfaces

Re: cluster fail

Lesley — Mon, 16 Dec 2024 01:08:09 GMT

I also spotted this couple days ago;

Looks like you have not defined eth2 as sync interface is this correct? Clusters needs to have a sync interface between 2 firewalls.

This indeed really needs to be checked and fixed.

Re: cluster fail

the_rock — Mon, 16 Dec 2024 01:19:17 GMT

I believe based on the last screenshot, it shows eth2 defined as sync, but something is clearly wrong, since the firewall still does not see it as such.

Andy