Failed to connect to a cluster member on SSL Remote Access VPN client

[Simon_Garay]

Hi guys,

I have ainadequate behaviour in the VPN client site, when the cluster is failover, all the user with MacOS, we see the message "USERNAME tried to connect, but you have reached the number of purchased licenses", and users with windows logean correctly.

In a moment suspect that the device standby, did not have its licenses correctly loaded, but validate them and they are correct.

FW1 - Standby (Problem):

=~=~=~=~=~=~=~=~=~=~=~= PuTTY log  =~=~=~=~=~=~=~=~=~=~=~=
cplic print
Host Expiration Features
X.X.X.21 never cpap-sg1540x cpsb-fw cpsm-c-2 cpsb-vpn cpsb-npm cpsb-logs cpsb-sslvpn-200 cpsb-ia cpsb-sslvpn-5 cpsb-adnc cpsb-apcl cpsb-ips cpsb-urlf cpsb-av cpsb-abot-l cpsb-aspm XX-XX-XX-XX-XX-XX-XX
X.X.X.21 never CPAP-SG1540X CPSB-FW CPSM-C-2 CPSB-VPN CPSB-NPM CPSB-LOGS CPSB-IA CPSB-SSLVPN-5 CPSB-ADNC CPSB-APCL CPSB-IPS CPSB-URLF CPSB-AV CPSB-ABOT-L CPSB-ASPM XX-XX-XX-XX-XX-XX-XX

Contract Coverage:

# ID Expiration SKU
===+===========+============+====================
1 | XXXXXXX | 24Apr2020 | CPSB-URLF-L-2Y
+-----------+------------+--------------------
|Covers: CPAP-SG1540X CPSB-FW CPSM-C-2 CPSB-VPN CPSB-NPM CPSB-LOGS CPSB-IA CPSB-SSLVPN-5 CPSB-ADNC CPSB-APCL CPSB-IPS CPSB-URLF CPSB-AV CPSB-ABOT-L CPSB-ASPM XX-XX-XX-XX-XX-XX-XX
| cpap-sg1540x cpsb-fw cpsm-c-2 cpsb-vpn cpsb-npm cpsb-logs cpsb-sslvpn-200 cpsb-ia cpsb-sslvpn-5 cpsb-adnc cpsb-apcl cpsb-ips cpsb-urlf cpsb-av cpsb-abot-l cpsb-aspm XX-XX-XX-XX-XX-XX-XX

FW2- Active:

=~=~=~=~=~=~=~=~=~=~=~= PuTTY log =~=~=~=~=~=~=~=~=~=~=~=
cplic print
Host Expiration Features
X.X.X.22 never cpap-sg1540x cpsb-fw cpsm-c-2 cpsb-vpn cpsb-npm cpsb-logs cpsb-sslvpn-200 cpsb-ia cpsb-sslvpn-5 cpsb-adnc cpsb-apcl cpsb-ips cpsb-urlf cpsb-av cpsb-abot-l cpsb-aspm XX-XX-XX-XX-XX-XX-XX
X.X.X.22 never CPAP-SG1540X CPSB-FW CPSM-C-2 CPSB-VPN CPSB-NPM CPSB-LOGS CPSB-IA CPSB-SSLVPN-5 CPSB-ADNC CPSB-APCL CPSB-IPS CPSB-URLF CPSB-AV CPSB-ABOT-L CPSB-ASPM XX-XX-XX-XX-XX-XX-XX
X.X.X.20 11Jul2017 CPSG-C-8-U CPSB-FW CPSB-VPN CPSB-IPSA CPSB-DLP CPSB-SSLVPN-U CPSB-IA CPSB-ADNC CPSG-VSX-25S CPSB-SWB CPSB-IPS CPSB-AV CPSB-URLF CPSB-ASPM CPSB-APCL CPSB-ABOT CPSB-CTNT CK-XX-XX-XX-XX-XX-XX-XX

Contract Coverage:

# ID Expiration SKU
===+===========+============+====================
1 | XXXXXXX | 24Apr2020 | CPSB-URLF-L-2Y
+-----------+------------+--------------------
|Covers: CPAP-SG1540X CPSB-FW CPSM-C-2 CPSB-VPN CPSB-NPM CPSB-LOGS CPSB-IA CPSB-SSLVPN-5 CPSB-ADNC CPSB-APCL CPSB-IPS CPSB-URLF CPSB-AV CPSB-ABOT-L CPSB-ASPM XX-XX-XX-XX-XX-XX-XX
| cpap-sg1540x cpsb-fw cpsm-c-2 cpsb-vpn cpsb-npm cpsb-logs cpsb-sslvpn-200 cpsb-ia cpsb-sslvpn-5 cpsb-adnc cpsb-apcl cpsb-ips cpsb-urlf cpsb-av cpsb-abot-l cpsb-aspm XX-XX-XX-XX-XX-XX-XX

In image appreciate that when rollback is executed, user can navigate again:

Log in, rollback:

look for information about it but I did not find, have you ever been presented with this problem?

Thank you very much for your support. 

[G_W_Albrecht]

I would advise you to weed out double / wrong licenses ! On the unit Standby (Problem), i find the same license X.X.X.21 CK XX-XX-XX-XX-XX-XX-XX two times, but once with cpsb-sslvpn-200 cpsb-sslvpn-5 and once with cpsb-sslvpn-5 only. Working FW2- Active has an additional Eval lic that ended 11Jul2017.

Please check the currently licensed blades in UserCenter, download the last generated license again, delete the installed licenses and only install the current ones!

[PhoneBoy]

Looks like you have cpsb-sslvpn-200 on one member but not the other.

And that will definitely cause the issue you're seeing.

Note that in general both/all cluster members should be licensed identically.

[Simon_Garay]

Thank you @Günther y @Dameon. 

We execute a debug vpn, we can see the log: 

available_om_licenses: number of connected users: om_users 18, snx_users -489, l2tp_users 0

We associate with sk120652

Symptoms

• Connection from SNX client / Capsule VPN client / Capsule Connect client / Endpoint Connect client to the VPN Cluster in High Availability mode fails with the error:
  You cannot receive an office Mode IP address because the security gateway does not have a license for Office mode. Contact your administrator.

• Restarting Check Point services ("cpstop;cpstart", reboot) on the Active cluster member resolves the issue until a fail-over occurs from the current Active cluster member to the Standby cluster member.

• Debug of VPND daemon (per sk89940) on the Active cluster member shows that the number of "snx_users" is negative:
  available_om_licenses: number of connected users: om_users XX, snx_users -YY, l2tp_users 0

• when a connection to the active member succeeds, kernel debug ('fw ctl debug -m VPN + warn') on the Standby cluster member shows:
  ;sslt_om_ip_params_post_sync: ERROR: Wrong # of vals XX;

• All licenses are valid and attached correctly to the Cluster Members.

Cause

Each time a Remote Access VPN client (SNX client / Capsule VPN client / Capsule Connect client / Endpoint Connect client) connects in SSL mode to a Cluster, its connection is synchronized to the Standby cluster member, but the counter of SSL users is not increased on the the Standby cluster member. However, when an SSL user disconnects, the counter of SSL users on the Standby cluster member is decreased.
Eventually, this leads to a wrong (negative) license count on the the Standby cluster member.
The issue becomes apparent after a failover from the current Active cluster member (that held the correct number of SSL users) to the Standby cluster member (on which the number of SSL users was not updated correctly).

After restarting services, we tested the connection with the vpn client, works correctly, connected to the affected cluster member.

[Cristian_gonzal]

good job !!

[G_W_Albrecht]

Hopefully you are on R80.10 already - then, Jumbo Take 142 will resolve that. For R77.30 you would have to ask TAC for the hotfix...

[Cristian_gonzal]

if we have 77.30 with hotfix 317, tac recommends installation hotfix take 338

to solve the problem

[G_W_Albrecht]

R77.30 jumbo take 338 does not list the issue from sk120652 as resolved (sk106162), only R80.10 Jumbo HotFix - Ongoing Take 142 (sk116380).

[Cristian_gonzal]

yes I know but the TAC at the time we raised the case with simon garay through the chat we mentioned that this hotfix 338 also mitigates the issue

[G_W_Albrecht]

I have added that as a commant in sk106162 !

[G_W_Albrecht]

SecureKnowledge solution ID: sk106162 and Title: "Jumbo Hotfix Accumulator for R77.30 (R77_30_jumbo_hf)" has now been edited based on your feedback.

Your feedback was:

------------------

R77.30 jumbo take 338 does not list the issue from sk120652 as resolved, that is listed only for R80.10 Jumbo HotFix - Ongoing Take 142 in sk116380. But according to TAC & simon garay, the jumbo hotfix 338 also mitigates the issue.

------------------

R&D responded: "Following issue from sk120652, raised below, I found that the issue (CR02447010) is listed under take_331 in R77.30 jhf sk106162." sk text was modified to mirror the text in R80.10 jumbo.

[Vladimir]

Simon,

Great troubleshooting feedback.

Do you mind renaming the original post so that it will be more descriptive of the root-cause issue?

Thank you.