R81.20 JHF Take 158 breaks Internet access and causes Application Control/HTTPS Inspection errors

[Mike_Jensen]

I am seeing a serious issue in my lab after installing R81.20 Jumbo Hotfix Accumulator Take 158 on a Check Point Security Gateway cluster.

The cluster was working normally before the upgrade. After installing Take 158 and rebooting the gateways, Internet access for systems behind the cluster stops working.

SmartConsole shows repeated errors related to Application Control, URL Filtering, and HTTPS Inspection. One example is:

Application Control - HTTPS Inspection error occurred

The same log also shows:

Internal system error in HTTPS Inspection

and:

Blocking request as configured in engine settings

The affected traffic is being blocked in the APC_URLF_Layer, rather than by a normal Access Control rule.

I also see this gateway status error:

Update failed. Contract entitlement check failed. Could not reach updates.checkpoint.com. Check DNS and Proxy configuration on the gateway.

DNS resolution from the gateway also fails. For example, nslookup does not complete successfully. This appears to be because the gateway itself can no longer reach the Internet, rather than simply an incorrect DNS configuration.

The behavior affects both:

Traffic passing through the gateway from internal clients
Connections initiated by the gateway itself, including connections to Check Point update services

Environment:

Check Point R81.20
ClusterXL Security Gateway cluster
Jumbo Hotfix Accumulator Take 158
Application Control and URL Filtering enabled
HTTPS Inspection enabled
Issue begins immediately after installing Take 158

The same policy and network configuration worked before the upgrade.

Most importantly, once I uninstall Take 158, the gateways immediately return to normal operation and Internet access is restored. I have been able to reproduce this behavior, which strongly suggests the issue is related to Take 158 rather than the existing policy or network configuration.

Has anyone else experienced Internet connectivity failures, URL categorization failures, or HTTPS Inspection internal errors after installing R81.20 Take 158?

I am particularly interested in whether this is a known Take 158 issue and whether there is a workaround other than uninstalling the take or changing Application Control/URL Filtering to fail open.

This is a lab environment, so I am not able to open a TAC case.

 

[PhoneBoy]

That suggest rad (a process) might be having issues.
Some troubleshooting is here: https://support.checkpoint.com/results/sk/sk183416

[jgriffiths]

Hi, we upgraded to R81.20 Take 158, after doing so we noted the management server was failing IPS updates amongst other things. 

I found the /etc/resolv.conf isn't being populated with the DNS servers or search domain despite the configuration existing in the GaiaUI and confirm via CLISH. I reapplied the configuration from the UI and confirmed /etc/resolv.conf is updated correctly. After doing so, DNS requests work and so do the updates. Unfortunately I rebooted the management server and found the configuration was removed once again.

I've raised this with our 3rd Party support. I hope this helps.

[Mike_Jensen]

Hi, Thank you for this information.  I will reinstall Take 158 and look at the resolv.conf file after.  I know when my environment was in the broken state I could see logs in SmartConsole for the SMS making DNS queries, but at the same time running nslookup updates.checkpoint.com on one of the gateways failed as it appeared it didn't have any DNS servers configured to try.

I will post here again with the results of my further testing.

Thanks again!

[Marco_Valenti]

would be nice to have an update from tac about this issue since this is going to be installed on a lot of gateway 

[Orel_Vanounou]

TAC is aware of this issue, which is manifested for some customers who upgraded (in-place upgrade) to R81.20 JHF 158 from an earlier version.

Workaround: (can be done before the upgrade to avoid the issue as well)

 
VSX: (from VS0, for SP use gclish)

clish> set dns mode default
clish> save config

NON-VSX:

[Expert@gw8120:0]# dbset resolv:mode default
[Expert@gw8120:0]# dbset :save
[Expert@gw8120:0]# reboot

Configure the DNS servers via clish/WebUI if necessary after that.

If any assistance is needed, please submit a Service Request to TAC with the related info (CPinfo, HCP Report).

[jgriffiths]

Thank you for the workaround, this has been applied and the DNS configuration is now persistent on reboot; however there are still some issues. Updatable objects are not being retrieved from the update server, the following error is displayed.

The Check Point Management server is unable to download the Updatable Objects package at this time.
This problem can be caused by:

1. No Internet connectivity.
2. No proxy server is defined.
3. Insufficient access permissions to the Internet/proxy.
4. Download content from Check Point is not allowed.

If this problem persists, contact support.

I've confirmed the management server can reach the update domains documented here - https://support.checkpoint.com/results/sk/sk83520

[Marco_Valenti]

that's pretty bad since updatable objects will break most of the internet traffic, thanks for sharing,let's cross finger for a new take

[Orel_Vanounou]

A solution and a workaround will be published in sk185210, that will be released soon.

[Bruno_Ramos]

Hello.

I'm having issues with R82 and the solution mentioned by you didn't work...

Any ideias?

Thank you

BR

[ShawnHu]

Are you install r82 take 118?

[Bruno_Ramos]

Hi.

Yes, the issues started after JHF 118 installation.

I had to revert to the previous take.

Thank you

BR