topic Re: ClusterXL and port 263 in Firewall and Security Management

ClusterXL and port 263

Don_Paterson — Fri, 08 Dec 2023 23:24:08 GMT

I noticed that an R81.20 HA ClusterXL (main train) didn't use port 256 to sync after a member reboot. It was not used at all.

Instead port 263 was used as the only port along with port 8116.

Is that the full sync port?

https://support.checkpoint.com/results/sk/sk52421

Clustering
TCP	263	not predefined	Full Synchronization between Security Group Members in a Scalable Platform (by CXLD daemons over the internal network 192.0.2.0/24)

Protocol	Port number	Service Name and Comment	Usage
Firewall
TCP	256	FW1 - Check Point Security Gateway Service	Connections to Security Gateway Service (to FWD daemon): Fetching topology information by Security Gateway (by FWD daemon) from Security Management Server or Domain Management Server (CMA) Full Synchronization between ClusterXL members (by FWD daemons) IPS packet capture Used by the following commands: fw isp_link <server_ip> fw getver <server_ip> fw logswitch -h <server_ip> fw lslogs <server_ip> fw fetchlogs <server_ip>
Clustering
TCP	256	FW1 - Check Point Security Gateway Service	Full Synchronization between ClusterXL members (by FWD daemons)

Re: ClusterXL and port 263

the_rock — Sat, 09 Dec 2023 02:15:07 GMT

My R81.20 cluster member

[Expert@CP-FW-02:0]# netstat -anp | grep 256
tcp 0 0 0.0.0.0:256 0.0.0.0:* LISTEN 8232/fwd
tcp 0 0 0.0.0.0:42569 0.0.0.0:* LISTEN 8232/fwd
[Expert@CP-FW-02:0]# netstat -anp | grep 263
tcp 0 0 0.0.0.0:263 0.0.0.0:* LISTEN 8189/cxld
unix 3 [ ] STREAM CONNECTED 26362 7986/sxl_statd
unix 3 [ ] STREAM CONNECTED 26364 7986/sxl_statd
unix 3 [ ] STREAM CONNECTED 26360 7986/sxl_statd
unix 3 [ ] STREAM CONNECTED 26363 7986/sxl_statd
unix 3 [ ] STREAM CONNECTED 26361 7986/sxl_statd
[Expert@CP-FW-02:0]#

Re: ClusterXL and port 263

Don_Paterson — Sat, 09 Dec 2023 07:00:37 GMT

What if you do a tcpdump for host FW-01 (other member) -w and then reboot the other member.

When it is back up then cpmonitor the capture file.

Re: ClusterXL and port 263

Don_Paterson — Sat, 09 Dec 2023 12:08:46 GMT

I did some debugs and I cannot see port 256 being part of the full sync.

I see a some evidence that 263 is the port used when the member wants to do a full sync.

[12516 3934414848]@FW-02[9 Dec 10:41:14] fwasync_get_maxbuf: maxbuf=4194304
fwsync: starting full sync with cluster member 10.10.10.2 (ctrl-c anytime to abort)

[12516 3934414848]@FW-02[9 Dec 10:41:14] fwsyncn_connect: server 10.10.10.2 alive 1

fwsyncn_connect: trying to connect to 10.10.10.2 with CXLD_PORT
[12516 3934414848]@FW-02[9 Dec 10:41:14] fwclient_do_connect_e: server 10.10.10.2 port 263 sicname CN=FW-01,O=xyz.abc.aic5ip cmd 57

[12516 3934414848]@FW-02[9 Dec 10:41:14] fwasync_conn_params_ex: fd: <24>, my addr: <10.10.10.3,61458>, peer addr: <10.10.10.2,263>

Re: ClusterXL and port 263

the_rock — Sat, 09 Dec 2023 12:50:26 GMT

As a matter of fact, did that last night and it was exact same outcome.

Best regards,

Andy

Re: ClusterXL and port 263

Don_Paterson — Sat, 09 Dec 2023 17:46:05 GMT

Can you say ElasticXL 😉

Just a couple of reference to make this post more interesting:

https://support.checkpoint.com/results/sk/sk180742

https://support.checkpoint.com/results/sk/sk97638

Security Gateway Software Blades and Features

Daemon	Section	Information
cxld	Description	Runs the cluster Full Sync (R81 and higher).
Path	$FWDIR/bin/cxld
Log file	$FWDIR/log/cxld.elg
To Stop	cpstop
To Start	cpstart
Debug	Runs with debug by default

Re: ClusterXL and port 263

Don_Paterson — Sat, 09 Dec 2023 18:27:29 GMT

From sk180742, this is a simple way to see full sync port 263 is used.

tailf $FWDIR/log/cxld.elg | grep 263

[cxld 9315 3934513920]@FW-01[9 Dec 18:51:35] fwasync_conn_params_ex: fd: <36>, my addr: <10.10.10.2,263>, peer addr: <10.10.10.3,35059>
[cxld 9315 3934513920]@FW-01[9 Dec 18:51:35] PM_session_init: given session I(CN=FW-01,O=SMS.abc.com.aizyx;CN=FW-02,O=SMS.abc.com.aizyx;263;syncn).
[cxld 9315 3934513920]@FW-01[9 Dec 18:51:35] PM_policy_query: input session I(CN=FW-01,O=SMS.abc.com.aizyx;CN=FW-02,O=SMS.abc.com.aizyx;263;syncn).

with grep -i instance it covers more detail on "full sync".

Note:
I used sk37030 debug steps as part of stopping and starting sync (forcing full sync) and included the tailf cxld.elg into that procedure to capture the output above.

fw ctl setsync off

fw -d fullsync 10.10.10.2 2

Re: ClusterXL and port 263

the_rock — Sat, 09 Dec 2023 18:49:04 GMT

I cant say elasticXL, as I have no clue in the world what that even is lol

Andy

Re: ClusterXL and port 263

the_rock — Sat, 09 Dec 2023 18:49:39 GMT

Yup, just did that tail command, nada, nothing

Andy

Re: ClusterXL and port 263

Don_Paterson — Sat, 09 Dec 2023 19:03:42 GMT

https://community.checkpoint.com/t5/General-Topics/R82-ElasticXL/m-p/194697

Re: ClusterXL and port 263

the_rock — Sat, 09 Dec 2023 19:05:14 GMT

Just looked it up before you gave the link...well, its definitely not applicable, since R82 is not even EA yet.

Andy

Re: ClusterXL and port 263

Don_Paterson — Sat, 09 Dec 2023 19:10:09 GMT

Try this in a separate session:

fw ctl setsync off

fw -d fullsync 10.10.10.2 2

I added note and sk reference to my post to cover that.

Re: ClusterXL and port 263

the_rock — Sat, 09 Dec 2023 19:20:40 GMT

Already done, no change. Only still see connection on port 256.

Re: ClusterXL and port 263

Don_Paterson — Sat, 09 Dec 2023 19:32:14 GMT

The port is used by the SP (Scalable Platform), as per sk52421.
Since TCP 256 was originally used and it seems there has been a change to 263 it may be that they are preparing for ElasticXL, or simply unifying sync mechanisms across platforms.

Re: ClusterXL and port 263

_Val_ — Mon, 11 Dec 2023 07:43:05 GMT

R82 is on EA for over 3 weeks: https://community.checkpoint.com/t5/Product-Announcements/R82-EA-Program-Production/ba-p/198695

Re: ClusterXL and port 263

_Val_ — Mon, 11 Dec 2023 07:48:02 GMT

Checking with the internal teams if the documentation needs to be updated. Will keep you posted.

Re: ClusterXL and port 263

Don_Paterson — Mon, 11 Dec 2023 07:51:16 GMT

Thanks Val.
I checked on an R80.40 cluster and there was no use of port 263, like in R81 and above. Only 256 was used.

Also, $FWDIR/log/cxld.elg did not exist on the R80.40 cluster member that I tested on.

Re: ClusterXL and port 263

_Val_ — Mon, 11 Dec 2023 12:34:57 GMT

We know. We expect documentation to be updated soon.

Re: ClusterXL and port 263

Timothy_Hall — Mon, 11 Dec 2023 14:37:39 GMT

Right I believe cxld was added in R81 to get the full sync process out of the kernel and into process space (or perhaps take that responsibility away from fwd), similarly to the Firewall Worker Instances being pulled out of the kernel with User Space Firewall (USFW) in fwk processes. Starting in R81.20 on the Quantum Lightspeed appliances only, even SecureXL is mostly located outside the kernel (called UPPAK). Eventually UPPAK will be the default mode for SecureXL on all appliances, very curious to see what kind of performance hit the fastpath will take as a result.

Re: ClusterXL and port 263

Martin_Raska — Mon, 11 Dec 2023 13:58:05 GMT

I have the same, R81.20

fwasync_conn_params_ex: fd: <34>, my addr: <192.168.254.1,263>, peer addr: <192.168.254.2,56792>
[cxld 30169 3933345856]@chkp-demo-gw-1[2 Dec 15:40:09] fwsyncn_init: starting new sync client on port 263