Error: Update failed. Contract entitlement check failed on VSX deployment

[SinisaZG]

A few days ago an error appeared on one of the two VSX gateways ( one is fine, no errors):

Status Failed (Anti-bot, anti-virus)
Description Update failed. Contract entitlement check failed. Could not reach "updates.checkpoint.com". Check DNS and Proxy configuration on the gateway.
Next update The next try will be within one hour

I have three virtual systems - an error is displayed on all of them.

I tried to reboot the VSX gateway several times  on which the problem is present - no luck

I tried to deinstall/install Anti-virus, Anti-bot - no  luck 

Output of command curl_cli -v -k https://updates.checkpoint.com/WebService/services/DownloadMetaDataService ;

* Trying 23.212.89.172...
* TCP_NODELAY set
* Connected to updates.checkpoint.com (23.212.89.172) port 443 (#0)
* ALPN, offering http/1.1
* *** Current date is: Fri Mar 1 13:11:27 2024
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* err is -1, detail is 2
* *** Current date is: Fri Mar 1 13:11:27 2024
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, [no content] (0):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, [no content] (0):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, [no content] (0):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, [no content] (0):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, [no content] (0):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* servercert: Activated
* servercert: CRL validation was disabled
* Server certificate:
* subject: CN=*.checkpoint.com
* start date: Dec 31 11:43:57 2023 GMT
* expire date: Jan 31 11:43:56 2025 GMT
* issuer: C=BE; O=GlobalSign nv-sa; CN=GlobalSign GCC R3 DV TLS CA 2020
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* servercert: Finished
* TLSv1.3 (OUT), TLS app data, [no content] (0):
* TLSv1.3 (IN), TLS handshake, [no content] (0):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, [no content] (0):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS app data, [no content] (0):
< HTTP/1.1 200 OK
< Content-Type: text/xml;charset=UTF-8
< Content-Length: 410
< Server: Apache-Coyote/1.1
< Date: Fri, 01 Mar 2024 12:11:27 GMT
< Connection: keep-alive

System is on R81.20, Take 38
I know there are a lot of posts like mine and  I have tried everything from similar posts listed

I'm out of ideas.....

[_Val_]

Looks like there is an issue with the certificate validation. Please open a TAC ticket for this.

[SinisaZG]

Thank you Val, I will do so.

[Lesley]

Are you talking about an error on VSX itself or on the VS?

[SinisaZG]

On VSX and on VS... 

[Lesley]

Is the error on gateway, mgmt or both?

[SinisaZG]

You got me thinking.... you mean when I try command  curl_cli -v -k ? 

I just tried to do this on a VSX GTW that is ok and  on a management server.... the output is the same on all three examples regarding  the certificate validation.

servercert: Activated
* servercert: CRL validation was disabled
* Server certificate:
* subject: CN=*.checkpoint.com
* start date: Dec 31 11:43:57 2023 GMT
* expire date: Jan 31 11:43:56 2025 GMT
* issuer: C=BE; O=GlobalSign nv-sa; CN=GlobalSign GCC R3 DV TLS CA 2020
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.

Issue is only with Anti-bot, Anti-virus Blades only on one  VSX gateway in Cluster ( other is fine)

[Lesley]

I mean the update failed error. You see it on gateway or mgmt or both? 

the certificate error is not related to this issue. 

also assume the issue is only on the standby member? If you do failover the problem then moves to other new standby member?

[SinisaZG]

Update failed error is related only to Anti-bot, Anti-virus Blades - Gateway Blades. Everything else is fine.

You are right. The issue is with the "standby" member, (it's not classic HA because of the VSLS and a customer who often uses its features.)

I will test with failover and try to see the results.... thanks for your reply.

[SinisaZG]

I did a failover, I tested with different options vsx_util vsls, admin up|down, cphastop|start.

Error is present only on one gateway (no matter if it is a active or standby) and is now present in two out of three virtual systems just for Anti-Virus Update Status . Anti-Bot Update Status is ok now.

So far it has been a problem with all three virtual systems (Anti-Virus&Anti-Bot)

Before failover I also tested connection with

# curl_cli -v -1 --cacert $CPDIR/conf/ca-bundle.crt https://updates.checkpoint.com

curl_cli -v -1 --cacert $CPDIR/conf/ca-bundle.crt https://secureupdates.checkpoint.com

curl_cli -v http://cws.checkpoint.com

https://support.checkpoint.com/results/sk/sk105757

and checked status of parameter 'fwha_forw_packet_to_not_active' and its set to 1

https://support.checkpoint.com/results/sk/sk43807

Right now it's quite confusing and I'm out of ideas so I am waiting for feedback from TAC

[the_rock]

I would say this might be worth TAC case, as you also rebooted the gateway, but no luck.

Best,

Andy

[SinisaZG · Accepted solution]

As we said, it was worth getting TAC involved...

I will try to be as clear as possible:

The TAC team could not find anything that would cause the issues. 
Before engaging the R&D team, we decided to update the system to the latest version first. (Take 38 was present.)
After updating the system, the error disappeared on all virtual systems except VS0, where it was replaced with the error "database version unknown" (IPS).
It has been confirmed that it is a bug in the system, and we are waiting for the R&D  team to make a hotfix.
Apparently, the same error occurs on older versions of the system, where it was solved by applying a hotfix.
In the meantime, while we were waiting for a hotfix from the R&D team, the problem resolved itself.
There are no more errors on the VSX system.

[the_rock]

Thanks for the update!

Andy