R82 elasticXL lab

[the_rock]

Hey boys and girls, ladies and gents,

I built R82 elasticXL lab and though I followed below link by @HeikoAnkenbrand , not sure if I cant make it work cause Im using eveNG or for what reason, but I created 2 separate elasticxl instances, but clustering part fails for some reason, so if anyone has an idea, happy to hear it 🙂

I could care less if this lab breaks, its super easy to rebuid anyway. 

This is the link I was referring to. I also attached some screenshots and outputs.

Andy

https://community.checkpoint.com/t5/Security-Gateways/R82-Install-ElasticXL-Cluster/td-p/206235

 

 

 

[Expert@CP-EXL-1-s01-01:0]# cphaprob state

Cluster Mode: HA Over LS

ID Unique Address Assigned Load State Name

1 (local) 192.0.2.1 100% ACTIVE(P) CP-EXL-1-s01-01

Active PNOTEs: None

Last member state change event:
Event Code: CLUS-114904
State change: ACTIVE(!) -> ACTIVE
Reason for state change: Reason for ACTIVE! alert has been resolved
Event time: Mon Jul 1 19:40:49 2024
[Expert@CP-EXL-1-s01-01:0]#

[Expert@CP-EXL-02-s01-01:0]# asg monitor
Mon Jul 01 20:44:20 EDT 2024

--------------------------------------------------------------------------------
| System Status - ElasticXL |
--------------------------------------------------------------------------------
| Up time | 39:27 minutes |
| Members | 1 / 1 |
| Version | R82 (Build Number 633) |
Mon Jul 01 20:44:21 EDT 2024
--------------------------------------------------------------------------------
| System Status - ElasticXL |
--------------------------------------------------------------------------------
| Up time | 39:29 minutes |
| Members | 1 / 1 |
| Version | R82 (Build Number 633) |
| FW Policy Date | 01Jul24 20:38 |
| AMW Policy Date | N/A |
--------------------------------------------------------------------------------
| Member ID Site1 |
| ACTIVE |
--------------------------------------------------------------------------------
| 1 ACTIVE |
--------------------------------------------------------------------------------

^C
[Expert@CP-EXL-02-s01-01:0]#

[Expert@CP-EXL-02-s01-01:0]# cphaprob -a if

CCP mode: Automatic

Interface Name: Status:

eth2 UP
eth3 UP
Sync (S) UP
magg1 (LS) UP

S - sync, HA/LS - bond type, LM - link monitor, P - probing

[Expert@CP-EXL-1-s01-01:0]#
[Expert@CP-EXL-1-s01-01:0]# cphaprob -a if

CCP mode: Automatic

Interface Name: Status:

eth2 UP
eth3 UP
Sync (S) UP
magg1 (LS) UP

S - sync, HA/LS - bond type, LM - link monitor, P - probing

Virtual cluster interfaces: 5

lo 127.0.0.1
eth2 192.168.10.238
eth3 169.254.0.238
Sync 192.0.2.1
magg1 172.16.10.238

[Expert@CP-EXL-1-s01-01:0]#

Virtual cluster interfaces: 5

lo 127.0.0.1
eth2 192.168.10.237
eth3 169.254.0.237
Sync 192.0.2.1
magg1 172.16.10.237

[Expert@CP-EXL-02-s01-01:0]#

And since elasticxl cluster object does NOT have an option to add cluster members, there is something obvious Im missing, but cant figure out what, so will check it later.

Andy

 

[emmap]

They need to be able to see each other over their Sync links (and it needs to have LLDP working as far as I know) and the second one should not be SIC'd to the management server as its own separate cluster if you want them to both be part of the same EXL gateway.

[the_rock]

Thank you. I may wipe out exl-02 tomorrow, re-crerate it again and see if I can sync them properly.

Andy

[the_rock]

I see that sync IPs are not pingable from either member, so thats 100% the issue. I will talk to one of my colleagues this week to see best way to make this work in eve-ng, as for regular cluster, its pretty simple, but same method does not work for eslasticxl sadly.

Andy

[the_rock]

FWIW, here is what I see on the FIRST one I installed:

[Expert@CP-EXL-1-s01-01:0]# cphaprob -a if

CCP mode: Automatic

Interface Name: Status:

eth2 UP
eth3 UP
Sync (S) UP
magg1 (LS) UP

S - sync, HA/LS - bond type, LM - link monitor, P - probing

Virtual cluster interfaces: 5

lo 127.0.0.1
eth2 192.168.10.238
eth3 10.254.10.238
Sync 192.0.2.1
magg1 172.16.10.238

[Expert@CP-EXL-1-s01-01:0]#

Then, 2nd one, which is not tied to the mgmt server, though for some odd reason. eth2 and 3 dont show up, though they definitely are enabled and on.

[Expert@CP-EXL-2-s01-01:0]# cphaprob -a if

CCP mode: Automatic

Interface Name: Status:

Sync (S) UP
magg1 (LS) UP

S - sync, HA/LS - bond type, LM - link monitor, P - probing

Virtual cluster interfaces: 5

lo 127.0.0.1
eth2 192.168.10.237
eth3 10.254.10.237
Sync 192.0.2.1
magg1 172.16.10.237

[Expert@CP-EXL-2-s01-01:0]#

[the_rock]

The only way to show same interfaces on 2nd member is to connect it to the mgmt server and install the policy, but that still does not change the fact cluster member cant be added to the 1st gateway.

I will check with our SE if this is expected or if there is any way to make this work with eve-ng.

Andy

[Expert@CP-EXL-2-s01-01:0]# cphaprob -a if

CCP mode: Automatic

Interface Name: Status:

eth2 UP
eth3 UP
Sync (S) UP
magg1 (LS) UP

S - sync, HA/LS - bond type, LM - link monitor, P - probing

Virtual cluster interfaces: 5

lo 127.0.0.1
eth2 192.168.10.237
eth3 10.254.10.237
Sync 192.0.2.1
magg1 172.16.10.237

[Expert@CP-EXL-2-s01-01:0]#

[Yair_Shahar]

Hi,

Once you have single member configure + configured as single gateway on smart console + SIC + Install Policy - then you can add other member to this ElasticXL cluster by WebUI or gclish > add cluster member.... (other members on same sync should be visible there)

[the_rock]

Hey @Yair_Shahar 

Thanks a lot for helping with this, I greatly appreciate it.

Just for the context, I followed EXACT process that Heiko had in the post I referenced, but I have a feeling something the way eve-ng works might be the issue here. So, below are things I tested:

1) followed Heiko's link, but got below message when trying to add:

[Global] CP-EXL-1-s01-01> add cluster member method request-id identifier 6e3077466f10d3d99db1f62254297612 site-id 1 format json
{
"response": 401,
"body": {
"message": "No info for request-id with value 6e3077466f10d3d99db1f62254297612",
"errors": "",
"code": "generic_error"
}
}
[Global] CP-EXL-1-s01-01>

2) I then reinstalled 2nd member, exact same issue

3) Once I connect 2 member to smart console and push policy, I see shows same sync, but I have NO CLUE where it comes from. Sorry, Im totally ignorant if you will when it comes to maestro, I know very basics of it, so apologies if these comments Im making sound stupid, but I see same thing on both members and as I mentioned to emmap, ONLY once both are connected to mgmt server, can I see same via cphaprob state, see below.

Andy

member 1:

[Expert@CP-EXL-1-s01-01:0]# cphaprob -a if

CCP mode: Automatic

Interface Name: Status:

eth2 UP
eth3 UP
Sync (S) UP
magg1 (LS) UP

S - sync, HA/LS - bond type, LM - link monitor, P - probing

Virtual cluster interfaces: 5

lo 127.0.0.1
eth2 192.168.10.238
eth3 10.254.10.238
Sync 192.0.2.1
magg1 172.16.10.238

[Expert@CP-EXL-1-s01-01:0]#

member 2:

[Expert@CP-EXL-2-s01-01:0]# cphaprob -a if

CCP mode: Automatic

Interface Name: Status:

eth2 UP
eth3 UP
Sync (S) UP
magg1 (LS) UP

S - sync, HA/LS - bond type, LM - link monitor, P - probing

Virtual cluster interfaces: 5

lo 127.0.0.1
eth2 192.168.10.237
eth3 10.254.10.237
Sync 192.0.2.1
magg1 172.16.10.237

[Expert@CP-EXL-2-s01-01:0]#

web UI shows same for both:

 

[Yair_Shahar]

it seems like you did FTW on the second member as well.

in ElasticXL - FTW should run only on first member (AKA SMO), rest of the members should be just installed without any additional direct step on them.

[the_rock]

Hey Yair,

Not sure what FTW means in this context, but keep in mind, when I did this yesterday, I litereally powered on R82 image, did NOT go through first time wizard, then tried adding that member on 1st cluster member, failed with error I gave. Are you saying I should install it again, go through wizard and NOT select part of elastic XL or do it totally different way?

Andy

[the_rock]

I think I get it now, sorry, not great with some abbreviations lol. I think FTW in this context means first time wizard, which I did NOT run yesterday when I did this, but again, error was exactly the same when trying to add a cluster member.

Andy

[ShaiF]

LLDP is not necessary. They are communicating using UDP broadcast packet over the Sync network

[the_rock]

@ShaiF @Yair_Shahar 

Just to make sure I got this right. So, should I delete 02 member from smart console, wipe it out, and then reinstall, go through wizard and NOT select part of elasticxl or select it and then try sync it?

Because again, I followed exact process Heiko gave in his initial link when adding a cluster member, it failed with error I provided and this was WITHOUT doing any initial config through web UI.

Andy

[Global] CP-EXL-1-s01-01> add cluster member method request-id identifier 6e3077466f10d3d99db1f62254297612 site-id 1 format json
{
"response": 401,
"body": {
"message": "No info for request-id with value 6e3077466f10d3d99db1f62254297612",
"errors": "",
"code": "generic_error"
}
}
[Global] CP-EXL-1-s01-01>

[Yair_Shahar]

not exactly

wipe out the second member, and reinstall it - that's it - do not run any FTW on it

[the_rock]

Right, which I did twice already and no matter what I do, always get below message : - (

Andy

[Global] CP-EXL-1-s01-01> add cluster member method request-id identifier 6e3077466f10d3d99db1f62254297612 site-id 1 format json
{
"response": 401,
"body": {
"message": "No info for request-id with value 6e3077466f10d3d99db1f62254297612",
"errors": "",
"code": "generic_error"
}
}
[Global] CP-EXL-1-s01-01>

[ShaiF]

before you add member. run from gclish 

> show cluster info provision

and see you see in output the other member in REQUEST_TO_JOIN state. i you do not see it you have issues on your Sync network (is it VM)?

only once you see it you can add it to the cluster

[the_rock]

Correct, its eve-ng platform. I just find it odd, as I never had sync issues with regular cluster in it, but this is obviously different. Give me 15-20 mins and I will update the thread.

Andy

[the_rock]

Sadly, still the same, BUT, since Im very persistent dude, I want to leave it in broken state, so can be fixed.

Andy

 

 

[ShaiF]

ok. now your second member is in clean install (we can see by the prompt)

since it kind of vm (but not vmware) so i guess we will need to do some WA.

Please share output of (from new member)

1. ifconfig -a

2. ps auxww | grep exl_detectiond

[the_rock]

Kind of vm, right, its eve-ng, so its considered vm, but not like say regular esxi. Btw, ONLY interface configured is eth0 with 192.168.1.1 IP, no static route, nothing yet.

Andy

 

[ShaiF · Accepted solution]

ok. since it's not officially supported product (the env-gn) do this WA:

From expert on the new member
1. vi /opt/ElasticXL/exl_detection/src/exl_detectiond.py

change this line 

if __machine_info.sync_ifn != 'Sync' and not __machine_info.is_vmware and not __machine_info.is_kvm:
to this:

if False: #__machine_info.sync_ifn != 'Sync' and not __machine_info.is_vmware and not __machine_info.is_kvm

2. run this:
dbset process:exl_detectiond t

dbset :save

tellpm process:exl_detectiond t

See ifconfig will show you eth1 as 192.0.2.254

try ping 192.0.2.1 and from SMO ping 192.0.2.254

[the_rock]

Hey Shai,

No sweat honestly, if its not supported, lets leave it alone, I dont like to waste time on unsupported things, plus, its not fair to you guys either. I tried, but no luck.

Thanks again for everything.

Andy

[the_rock]

Btw, wanted to say, I left first one I created there for testing, so thats totally fine. 

Andy

[the_rock]

Hey @ShaiF 

Thanks again so much for this process, I did make it work doing so, I suppose I missed a character or something else first time I did it.

Really grateful for the help!

Andy

[Arne_Boettger]

Thanks for sharing this workaround - I would appreciate if there was a sincere warning for unsupported deployments, but an obvious way to install it anyway for labs and test environments.

In addition: I needed to apply the Workaround to BOTH gateways to be able to establish a cluster in my Proxmox lab.

[the_rock]

That would be nice warning, agree.

[root]

Thank you for the solution and it works for me in eve-ng

Few tips for running in eve ng

- eve ng template can't be use for the member fw, that will keep showing the same request id all the new Member and it will fail during the cluster add, you have to install from ISO separately for the new members.

- the python trick (from ShaiF), need to run on both members and SMO.

Finally It's nice to see ElasticXL running in my eve-ng.

[the_rock]

Nice to know, thanks for that!

Andy

[Yasushi_Kono1]

Hi ShaiF,

after having configured everything as described, I have got the 192.0.2.254 address on eth1. However, if I insert the command
add cluster member method request-id identifier xxxxxxxxxxxxxxxxxxxxxxxx site-id 1 format json

then I get the message info: 
"message" : "No info for request-id with value xxxxxxxxxxxxxxxxxxxxxxxxxxxx", 

How could this problem be resolved? Any hints?

Thanks in advance!

I try to do the configuration on a cloud-based lab environment specifically desgined for our CP classes (skillable).

[ShaiF]

Hi @Yasushi_Kono1,

have you tried the simpler way to add member using hostname /  serial-number?

can you share the output of 'show cluster info provisioning' or 'show cluster info" from gclish?
BTW:  you do not need to put XXX on the request-id  value as it is public key, no one can do nothing with it and it meant to be exposed. the private key is on the member you want to join so he's the only one who can join the cluster if you are using request-id as method
Regards,

Shai.