R82 ElasticXL & VSNext Issues

[genisis__]

Built a R82 ElasticXL & VSNext Lab in Proxmox, JHFA installed is Take 25.

- I managed to delete ID0/VS0 which I should never be able to do.

- reassign magg1 from vs500 so in affect knocking at management, again should not be able to do this.

If you have had similar issues please post and hopefully Checkpoint can review and respond. 

I had one other issue where I created a virtual gateway through gclish however for some reason I could not connect to the management interface on the VG. So attempted to delete via GUI and then the system crashed.  I've logged a TAC case to investigate the crash logs.

[_Val_]

You need one eval license per GW in the cluster, no way around it.

[genisis__]

Val - When I applied the wrong license to the active node, it replicated it to the standby node, so I can't see how I could apply a separate license to it (Although it makes sense what your saying, I don't see how you can do this unless your running traditional ClusterXL / VSX).

That does lead to another question, on how on earth you can run active/standby Nodes but ensure VNext gateways can be distributed among nodes (I believe that is possible from a a specific JHFA release).
Also if license are replicated then does that mean you should only have to purchase licenses for 1 node (something not clicking here).

[_Val_]

As Emma mentioned, you need to use local licenses. 

[genisis__]

I actually did apply a local license, but I used 'cplic put', removed the license from both nodes (note: the license was replicated).

the repeated added the license to the active node, but this time using 'g_cplic put',  I then had to push policy.

The ran to another issue of interface mismatch.

tp_dummy_5 99.81.112.231 <-- This was on the active unit, but not on the standby, and cphaprob did not like this.

I removed the ZeroPhishing blade and this disappeared and fixed the issue.

cphaprob status now looks like this:

1 (local) 192.0.2.1 100% ACTIVE(P) FW-s01-01
15 192.0.2.15 100% ACTIVE FW-s02-01

I think this needs to be looked at, additionally the nodes where failing over when the status changed at the VS level.

[genisis__]

Now this seems stable again, next task I want to do is uninstall JHFA_19 and install JHFA_33. 

I have uploaded the image to the active node, but how would I upload to the standby? 

Is there a specific installer command I use to upload to node 2_1 from 1_1?

[Bob_Zimmerman]

CPUSE should handle that. It has a special mode for Maestro and ElasticXL. When you import a package, it should copy it to all members and import it on all members.

[genisis__]

Got this error when attempting to uninstall a Jumbo from Standby node (See sk182043)

Update Service Engine
+------------------------------------------------------------------------------+
|Member ID |Status |
+-------------+----------------------------------------------------------------+
|2_01 |Uninstalling package... |
+-------------+----------------------------------------------------------------+

[ERROR] Failed to pull DB data.

-----

[Global] FW-s01-01:0> show installer packages all

Querying cluster members...

1_01:
Display name Status
R82 Jumbo Hotfix Accumulator Recommended Jumbo Take 19 Installed
R82 Jumbo Hotfix Accumulator Take 33 Imported

2_01:
Display name Status
R82 Jumbo Hotfix Accumulator Recommended Jumbo Take 19 Installed
R82 Jumbo Hotfix Accumulator Take 33 Downloaded

So not really sure what to make of this?  I'm expecting the jumbo to be uninstalling, but its doing nothing. The SK is indicating the message I had is a cosmetic issue (Note: running CPUSE Agent 2589 on both nodes).

[CEEJAY]

Hello, have you fixed the problem? I am encountering the same problem when uninstalling the Hotfix. Thanks

[_Val_]

Probably the best is to open a support ticket with TAC

[genisis__]

I'm actually about to jump on a call with TAC as they want to test in my lab.

[_Val_]

According to sk182043, this is a cosmetic issue with the Deployment Agent. See the mentioned SK for the workaround. If you are still stuck with it, open a support ticket with TAC.

[CEEJAY]

Based on sk182043, this is a cosmetic issue, does it mean that if I uninstall the hotfix, it will prompt the error in the cli, but it in reality the uninstallation is successful? 

[_Val_]

Please open a TAC case, because it might actually affect the uninstall process. Check with them.

[genisis__]

So had part 1 of a TAC session - and with the update CPUSE Agent 2589 there has been an improvement, but still there is an issue.

TAC are still investigating.

They confirmed how to transfer files between nodes as well:

Assuming we have a single node in each site, and configured for two sites we would have the followinf node IDs : 1_01 & 2_01

If I want to copy a file from node 1_01 to 2_01 I would do the following as an example:

asg cp2blades -b 2_01 /var/log/tmp/dummyfile.txt

[the_rock]

Thanks for that @genisis__ 

[CEEJAY]

How bout if there are two nodes in one site? 1_01 & 1_02. For example, I transfer a file to local of 1_01 via WinSCP. Does the file will be copied on local of 1_02 since there is an auto cloning feature? TIA.

[Bob_Zimmerman]

No. It might be copied if you reboot the member and the member decides it needs to copy the latest lightshot from another member, but I would not depend on this.

Importing a CPUSE package is a special case. CPUSE distributes the package to the members internally. This won't work for arbitrary files, only for CPUSE packages.

[genisis__]

Was just thinking  (Bad Idea I know!),  if we have a ElasticXL / VSNext setup, how is 'local.arp' handled if using manual NAT?  I would assume we should not be making any changes to the standby node as its a clone, yet I would have thought the mac's would still be different?

[Bob_Zimmerman]

That's a good question. I tried adding a proxy ARP entry to a VS in gclish, and it took the config (unlike on older VSX), but the proxy ARP entry doesn't actually show up in 'fw ctl arp'.

The members use predictable virtual MACs. For example, my s01-01 bond2 is 00:1c:7f:81:02:fc, and s01-02 bond2 is 00:1c:7f:81:42:fc. I don't know offhand if that changes based on which one is acting as the pivot member.

[Alex-]

This seems to be documented in sk30197.

[Bob_Zimmerman]

Unfortunately, the documentation is wrong.

[Expert@DallasticXL-s01-01:1]# gclish -c "show arp proxy all"
1_01:
IP Address              MAC Address / Interface         Real IP Address         
172.16.120.15           bond2.120                       172.16.120.1            

1_02:
IP Address              MAC Address / Interface         Real IP Address         
172.16.120.15           bond2.120                       172.16.120.1            


[Expert@DallasticXL-s01-01:1]# local_arp_update
The script is not supported in ElasticXL

[Expert@DallasticXL-s01-01:1]# g_allc cat $FWDIR/conf/local.arp
-*- 2 blades: 1_01 1_02 -*-
cat: /opt/CPsuite-R82/fw1/CTX/CTX00001/conf/local.arp: No such file or directory

[Expert@DallasticXL-s01-01:1]# asg_local_arp_verifier
-bash: asg_local_arp_verifier: command not found

[Expert@DallasticXL-s01-01:1]# g_fw ctl arp
-*- 2 blades: 1_01 1_02 -*-
No proxy ARP entries

[emmap]

OK the instructions there are a bit wrong. I tried on my EXL cluster (no VSNext to try here at the moment unfortunately) and got this result:

[Global] EXLGW-s01-01> add arp proxy ipv4-address 10.1.2.243 interface magg1 real-ipv4-address 10.1.2.254
1_01:
success

1_02:
success

[Expert@EXLGW-s01-01:0]# g_cat $FWDIR/conf/local.arp
-*- 1 blade: 1_01 -*-
# This file was AUTOMATICALLY GENERATED
# DO NOT EDIT
# Please use Gaia Portal or clish command to configure ARP proxy
10.1.2.243 00:1c:7f:8d:36:c7 10.1.2.254
-*- 1 blade: 1_02 -*-
# This file was AUTOMATICALLY GENERATED
# DO NOT EDIT
# Please use Gaia Portal or clish command to configure ARP proxy
10.1.2.243 00:1c:7f:8d:36:63 10.1.2.254

 [Policy Install]

[Expert@EXLGW-s01-01:0]# g_fw ctl arp
-*- 1 blade: 1_01 -*-
(10.1.2.243) at 00-1c-7f-8d-36-c7 interface 10.1.2.254
-*- 1 blade: 1_02 -*-
(10.1.2.243) at 00-1c-7f-8d-36-63 interface 10.1.2.254

 I'm not sure why your arp file isn't updating, but can you do the policy install before checking the arps?

[genisis__]

Will the SK get corrected?

[emmap]

I'll see about getting it updated. 

[_Val_]

@emmap and all, I already raised an internal task for that, but please go ahead.

[genisis__]

thanks.

[Bob_Zimmerman]

I added the entry on the command line, then installed policy on VS1 (where I added the proxy ARP entry) and VS0 (just because). I also ensured I had the "Merge manual and automated proxy ARP" box checked ahead of time. I've also rebooted both members just to be absolutely sure.

I was mostly commenting on how the documentation says to run a command which apparently doesn't support ElasticXL, and another command which doesn't exist on an ElasticXL system. I expect the actual proxy ARP issue is something simple.

[Sergei_Shir]

The relevant commands were updated (you will see the changes ~15 minutes from now)

https://support.checkpoint.com/results/sk/sk30197

Procedure for Scalable Platforms (Maestro / Scalable Chassis / ElasticXL)

[Bob_Zimmerman]

I just updated my cluster and management to jumbo 41 and tried the new documentation. Before all these commands, I have enabled "Merge manual and automated proxy ARP" and pushed policy. The push was successful.

[Expert@DallasticXL-s01-01:0]# vsenv 1
Context is set to Virtual Gateway DallasticVS1 (ID 1).
[Expert@DallasticXL-s01-01:1]# cpinfo -y fw1 | grep MAIN

This is Check Point CPinfo Build 914000250 for GAIA
	HOTFIX_R82_JUMBO_HF_MAIN	Take:  41
[Expert@DallasticXL-s01-01:1]# gclish
[Global] DallasticXL-s01-01:1> delete arp proxy ipv4-address 172.16.120.15
1_01:
success

1_02:
success

[Global] DallasticXL-s01-01:1> add arp proxy ipv4-address 172.16.120.15 interface bond2.120 real-ipv4-address 172.16.120.1
Interfaces which don't appear may belong to other VS's
1_01:
Interfaces which don't appear may belong to other VS's

1_02:
Interfaces which don't appear may belong to other VS's

[Global] DallasticXL-s01-01:1> save config
1_01:
success

1_02:
success

[Global] DallasticXL-s01-01:1> show arp proxy all
1_01:
IP Address              MAC Address / Interface         Real IP Address         
172.16.120.15           bond2.120                       172.16.120.1            


1_02:
IP Address              MAC Address / Interface         Real IP Address         
172.16.120.15           bond2.120                       172.16.120.1            


[Global] DallasticXL-s01-01:1> exit
[Expert@DallasticXL-s01-01:1]# asg_cp2blades $FWDIR/conf/local.arp
Source: /opt/CPsuite-R82/fw1/CTX/CTX00001/conf/local.arp does not exist.
Operation aborted
[Expert@DallasticXL-s01-01:1]# g_allc cat $FWDIR/conf/local.arp
-*- 2 blades: 1_01 1_02 -*-
cat: /opt/CPsuite-R82/fw1/CTX/CTX00001/conf/local.arp: No such file or directory

So it looks like adding the proxy ARP entry in clish doesn't create local.arp for the VS.

I ran 'touch $FWDIR/conf/local.arp', confirmed /opt/CPsuite-R82/fw1/CTX/CTX00001/conf/local.arp exists, rebooted, and tried everything again. Now the file exists, but it's empty:

[Expert@DallasticXL-s01-01:0]# vsenv 1
Context is set to Virtual Gateway DallasticVS1 (ID 1).
[Expert@DallasticXL-s01-01:1]# gclish
[Global] DallasticXL-s01-01:1> delete arp proxy ipv4-address 172.16.120.15
1_01:
success

1_02:
success

[Global] DallasticXL-s01-01:1> add arp proxy ipv4-address 172.16.120.15 interface bond2.120 real-ipv4-address 172.16.120.1
Interfaces which don't appear may belong to other VS's
1_01:
Interfaces which don't appear may belong to other VS's

1_02:
Interfaces which don't appear may belong to other VS's

[Global] DallasticXL-s01-01:1> save config
1_01:
success

1_02:
success

[Global] DallasticXL-s01-01:1> show arp proxy all
1_01:
IP Address              MAC Address / Interface         Real IP Address         
172.16.120.15           bond2.120                       172.16.120.1            


1_02:
IP Address              MAC Address / Interface         Real IP Address         
172.16.120.15           bond2.120                       172.16.120.1            


[Global] DallasticXL-s01-01:1> exit
[Expert@DallasticXL-s01-01:1]# asg_cp2blades $FWDIR/conf/local.arp
100% /var/opt/CPsuite-R82/fw1/CTX/CTX00001/conf/local.arp
Operation completed successfully
[Expert@DallasticXL-s01-01:1]# g_allc cat $FWDIR/conf/local.arp
Command completed successfully

[Expert@DallasticXL-s01-01:1]# cat $FWDIR/conf/local.arp
[Expert@DallasticXL-s01-01:1]# cat /opt/CPsuite-R82/fw1/CTX/CTX00001/conf/local.arp 
[Expert@DallasticXL-s01-01:1]# 

[_Val_]

That SK is now fixed and should have the correct instructions for local ARP definitions for Maestro and ElasticXL