Not able to configure VSX cluster

[robertp]

Hey. We bought 4 new devices (9400) and are trying to configure two new VSX clusters. I enabled clusterxl and vsls in cpconfig, configured ntp, default route, hostnames and pretty much that's it. When adding the cluster to a domain in MDS throws me an error (see attached). I am running r81.20 with jumbo 118. MDS is also r81.20 and take 113. 

Any ideas, suggestions? There are no interface aliases so sk144832 is not valid.

Of course I raised a case with TAC but they refused to help saying I need to pay for professional services (which for me is absolutely absurd) as it's a new deployment.

[Chris_Atkinson]

Can you share the routing table and interface ip/netmask etc?

[robertp]

[Expert@redacted:0]# ifconfig
Mgmt Link encap:Ethernet HWaddr 00:1C:7F:C7:E2:EA
inet addr:10.43.16.24 Bcast:10.43.16.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:362753 errors:0 dropped:0 overruns:0 frame:0
TX packets:30189 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:22495554 (21.4 MiB) TX bytes:4468455 (4.2 MiB)

Sync Link encap:Ethernet HWaddr 00:1C:7F:C7:E2:E9
inet addr:100.100.100.1 Bcast:100.100.100.3 Mask:255.255.255.252
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

eth5 Link encap:Ethernet HWaddr 00:1C:7F:C7:E2:E5
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:9354 errors:0 dropped:0 overruns:0 frame:0
TX packets:5643 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2124930 (2.0 MiB) TX bytes:1929906 (1.8 MiB)

lo Link encap:Local Loopback Media:unknown(auto)
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK NOTRAILERS RUNNING PROMISC ALLMULTI DYNAMIC MTU:65536 Metric:1
RX packets:3303499 errors:0 dropped:0 overruns:0 frame:0
TX packets:3303499 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:639903185 (610.2 MiB) TX bytes:639903185 (610.2 MiB)

[Expert@redacted:0]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.43.16.1 0.0.0.0 UG 0 0 0 Mgmt
10.43.16.0 0.0.0.0 255.255.255.0 U 0 0 0 Mgmt
100.100.100.0 0.0.0.0 255.255.255.252 U 0 0 0 Sync
redacted 10.43.16.1 255.255.0.0 UG 0 0 0 Mgmt -- I added one of our mgmt networks to check if it will fix it, same result when it wasn't here

[Lesley]

clish -> set vsx off

make screenshots of the interface in gaia web interface and also routing table.

screenshot default route, check if there is no second ip configured as default ip. 

after you done turn vsx on

route -n is from the linux OS. Would like to see the gaia config. alternative is clish -> show configuration

[robertp]

Hey, doesn't look suspicious:

redacted> set vsx off
redacted> show configuration static-route
set static-route default nexthop gateway address 10.43.16.1 on
set static-route 161.89.0.0/16 nexthop gateway address 10.43.16.1 on
redacted> show route
Codes: C - Connected, S - Static, R - RIP, B - BGP (D - Default),
O - OSPF IntraArea (IA - InterArea, E - External, N - NSSA),
IS - IS-IS (L1 - Level 1, L2 - Level 2, IA - InterArea, E - External),
A - Aggregate, K - Kernel Remnant, H - Hidden, P - Suppressed,
NP - NAT Pool, U - Unreachable, i - Inactive

S 0.0.0.0/0 via 10.43.16.1, Mgmt, cost 0, age 407222
C 10.43.16.0/24 is directly connected, Mgmt
C 100.100.100.0/30 is directly connected, Sync
Sync
C 127.0.0.0/8 is directly connected, lo
S redacted/16 via 10.43.16.1, Mgmt, cost 0, age 407222

[genisis__]

what version are you running?  Do you actually have VSX licenses for all 4 nodes?

i normally build gateways as  normal, ensure you have defined your sync networks and your layer setup is spot on.

The create a VSX cluster in the DMS you want to manage the cluster (normally I do it on the Main DMS), and let the process convert the gateways to VSX.

[robertp]

Hi, 81.20. I don't have the licenses on yet but I'm still on the trial one for a few days so that should cover it. I know I won't be able to create the needed VS-es but I would expect to be able to add the cluster at least.

As for creating the cluster - that's what I did at start. Also tried enabling VSX first - no difference.

[AkosBakos]

I am not 100% sure about that, the trial license contains the VSX feaute or not?

can you paste the #cplic print -p output?

[genisis__]

I believe it does - I did a lab in Proxmox and used a trial license and that seemed to work fine.

[robertp]

redacted> cplic print -p
Host Expiration Primitive-Features

======================================================================
Check Point product trial period will expire in 10 days.
Until then, you will be able to use the complete Check Point Product Suite.
Please obtain a permanent license from Check Point User Center at:
https://usercenter.checkpoint.com/pub/usercenter/get_started.html
======================================================================

But as genisis__ mentioned - it should work fine. This is also what I did the last time I set up a VSX. The only thing missing is the VS license (I even remember reading that 2 VS (root+1 production) are always available)

[genisis__]

On my trial license (when it was active) It listed all the features.

How did you generate the trial license?  Did you you select all-in-one?  So everything is included? (Would do a screen shot but Checkpoint site is down again).

[robertp]

Hi, It's the one that comes in with the box, I didn't generate a new one. I think that one always showed like this. 

In the meanwhile, I tried adding a single VSX gateway instead of the cluster and it still doesn't work, but the error is different. Maybe that will be a better clue.

[AkosBakos]

The trial licence created when you install the  component.

The EVAL license that what what are you talking about 🙂

A

[robertp]

Eval, right 😉 Still I think it should work. I'll add the proper licenses just for the sake of it but I doubt that will fix it (still it will be funny if it does).

[AkosBakos]

Have you checked this SK?

"Failed to resolve Management Virtual System NIC" error

https://support.checkpoint.com/results/sk/sk92556

[robertp]

Hi, saw it but neither the platform nor the version match. The naming convention of the Mgmt interface also.

[genisis__]

Got into the support site, finally. 

The 'Eval' I generated was "All In One Security Bundle"

[AkosBakos]

You told that, this method worked in your ALB environment. I suppose tht, you install a brand new MGMT for testing.

We ran into not the same error, but a little bit similar. The root cause was thatm the implied rules were not allowed.

Compare the Global parameters between you LAB and PROD environment. Maythe turns out something.

A

[genisis__]

I've had the problem as well - the Implied rules defo need to be enabled prior to VSX cluster setup

[robertp]

I deployed a new CMA for this and left everything default. Implied rules are definitely enabled, unless I am missing something obvious. 

I also attached the license to one of the boxes I am testing on right now:

[Expert@redacted:0]# cplic print
Host Expiration Features
10.43.16.24 never CPAP-SG940X CPSB-FW CPSG-C-16-U CPSG-C-4-U CPSM-C-2 CPSB-VPN CPSB-NPM CPSB-LOGS CPSB-IA CPSB-SSLVPN-5 CPSB-ADNC CPSG-VSX-3S CPSB-IPS CPSB-URLF CPSB-APCL CPSB-AV CPSB-ABOT-L CPSB-ASPM CPSB-CTNT redacted

[genisis__]

So your licensed for x3 VS's (Not sure what the 'S' at the end means)

[robertp]

Correct, I just need root + 2 VS. Doesn't really matter at this point as I can't even set up the gateways 😞

[robertp]

I just reverted one of the devices to defaults. Configured only the mgmt interface and default route in cli (removed the 'default' default route of course). In first time wizard - hostname, dns, ntp, sic. No clustering enabled, just trying to add the single VSX gateway - same result as previously. If it's not a bug in the 9400 platform then it has to be a bug on the MDS...

[genisis__]

Suggest a TAC case - as this does not sound right, to narrow it down, perhaps spin up a new manager  in virtualization tool of choice and try from there to narrow it down.

Also on the 9400 I assume you installed R82 with the latest Recommended Jumbo (HFA43)?

[robertp]

Hi, R81.20 with take 118. Also tried naked 81.20. Our MDS is still on R81.20.

I got a TAC case but they refuse to pick it up saying I need to buy PS support. Escalating it now.

[genisis__]

That does seem really strange, and its seems like a real issue rather then configuration.

Can we eliminate the MDS and you just build a simple Management VM and try it on there?

[robertp]

In case anyone finds it in the future and has the same issue. Reading the documentation carefully helps. NAT is not supported BOTH for gateways and MDS with VSX (at least for R81.20 and lower). My MDS was reachable with original addresses but the gateways were behind NAT.

Works fine after deploying a separate SMS in the customer mgmt network where NAT isn't needed.

[Bob_Zimmerman]

The S at the end is for "small". The "CPSG-VSX-3S" can only be added to "small" gateways. I forget where the line is, but you definitely can't put it on a five-digit box. I want to say the small versions of licenses and subscriptions are cheaper than the medium or large versions.