topic Re: Categorize HTTPS Websites in Firewall and Security Management

Categorize HTTPS Websites

Nandhakumar — Fri, 19 Nov 2021 16:07:26 GMT

After I enabled "categorize https websites" on my internet facing gateway, some of the government specific websites stopped working and rest of the internet sites worked well. Any idea what would went wrong?

I haven't seen any drops in smart console during the issue reported time.

Re: Categorize HTTPS Websites

the_rock — Fri, 19 Nov 2021 16:33:33 GMT

Keep in mind one thing...that setting, in all honesty, is totally useless if you dont do HTTPS inspection. Do you have inspection enabled? Ok, maybe thats wrong term, its not useless, but wont help you much with categorizing anything. If you have url filtering enabled, are you blocking that specific category?

Re: Categorize HTTPS Websites

_Val_ — Fri, 19 Nov 2021 16:44:15 GMT

Not true. With R80.40 and up, SNI enforcement and categorization is used even if HTTPSi is off.

Re: Categorize HTTPS Websites

the_rock — Fri, 19 Nov 2021 16:47:23 GMT

O really? Hm, I did not know that, my bad...I remember once though working with customer and TAC and they did not have https inspection enabled and TAC guy said in that case categorize https sites was not going to change anything at all. Customer was on R80.40, cant remember what jumbo hotfix though.

Re: Categorize HTTPS Websites

Nandhakumar — Sat, 20 Nov 2021 04:46:20 GMT

We don't have https inspection enabled. Our gateway is running with Gaia OS R81 and Jumbo Take 23. Then why still its not working?

Can you share what are all the things that will be checked, if we enable "Categorize HTTPS websites" option?

Re: Categorize HTTPS Websites

mcatanzaro — Sat, 20 Nov 2021 07:07:45 GMT

What does the browser show on these now non-functioning websites?

I.e. any specific errors seen?

Re: Categorize HTTPS Websites

Nandhakumar — Sat, 20 Nov 2021 08:07:08 GMT

No error/drop was seen in smart console logs as well as in browser. These sites have been accessed via SAP application.

I want to know working principle for "Categorize HTTPS website" option?

Re: Categorize HTTPS Websites

_Val_ — Sat, 20 Nov 2021 10:41:50 GMT

That statement would only be true for R80.30 and below. In R80.40 and up there is SNI verification tech, which sends independent TLS request from a security GW, to verify SNI and a site certificate are matching. It is called sometimes "SSL Lite" and allows us to categorise sites even if HTTPSi is inactive.

We have also publicised this functionality quite a lot in 2019 🙂

Re: Categorize HTTPS Websites

_Val_ — Sat, 20 Nov 2021 10:46:52 GMT

As already mentioned above, after enabling this option, GW would try retrieving and reading site TLS certificate. Please check your GW can successfully resolve and connect to the mentioned sites directly. curl_cli and nslookup would provide you with the answers.

Re: Categorize HTTPS Websites

Nandhakumar — Sat, 20 Nov 2021 11:52:20 GMT

I am not sure what to look specifically in curl_cli output? Can you guide if we need to look specifically something?

Below is the curl command output from the gateway

[Expert@GW:0]# curl_cli https://www.altinn.no
curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

[Expert@GW:0]# curl_cli https://www.virustotal.com/gui/home/upload
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

[Expert@GW:0]# curl_cli https://urlcat.checkpoint.com/urlcat/main.htm
curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.haxx.se/docs/sslcerts.html

Re: Categorize HTTPS Websites

the_rock — Sat, 20 Nov 2021 12:35:44 GMT

This post may also be helpful, though I believe this was before R80.40, but phoneboy gave great explanation. I was not aware that those changes @_Val_ mentioned were implemented in newer versions...

https://community.checkpoint.com/t5/Management/Difference-between-HTTPS-Inspection-and-Categorize-HTTPS/td-p/20073

Re: Categorize HTTPS Websites

Nandhakumar — Sat, 20 Nov 2021 13:31:14 GMT

I am not getting 100% context in that thread. Not sure what might be issue here? How to fix the same?

Does "Categorize HTTPS websites" will inspect encrypted traffic from client machine? If no, then what it will do?

For sure, the site worked well if I un-tick "Categorize HTTPS Websites" option.

Can we enable exception only for specific sites that should not categorize with https?

Also, does it have limitation with SSL site Grade. Only works for sites with the specific grades?

Re: Categorize HTTPS Websites

the_rock — Sat, 20 Nov 2021 14:03:26 GMT

Maybe opening TAC case would not be a bad idea...below is section from documentation about it:

Application and URL Filtering - Advanced Settings - General (checkpoint.com)

So, it really boils down to whether certificate is trusted or not, because based on that, the categorization would work differently...

Categorize HTTP sites

This option lets Application and URL Filtering assign categories to HTTPS sites without activating HTTPS inspection. It assigns a site category based on its domain name and whether the site has a valid certificate. If the server certificate is:

Trusted - Application and URL Filtering gets the domain name from the certificate and uses it to categorize the site.
Not Trusted - Application and URL Filtering assigns a category based on the IP address.

Application and URL Filtering uses these pages (in the SmartConsole Manage & Settings tab > Blades > HTTPS Inspection > Configure in SmartDashboard) to make sure that a certificate is valid:

Trusted CAs page - Makes sure the certificate is not stolen or revoked.

Note : If your company issues certificates, you must add your company CA to the list of Trusted CAs.

HTTPS Validation page - If the certificate is blacklisted, for example, it is not trusted and the site categorized according to its IP address.

Important - Application and URL Filtering gets the site URL from the SSL "CONNECT" request sent to the proxy, if:

There is a proxy between the Firewall and the destination site or
The Firewall is configured to work as a proxy

Re: Categorize HTTPS Websites

Timothy_Hall — Sat, 20 Nov 2021 18:07:26 GMT

A few points:

1) As mentioned R80.40 and later (and R80.20/R80.30 with latest Jumbo HFA) look at the SNI when making a categorization decision instead of just the site name on the certificate, but it is highly unlikely this operation is the cause of your issue.

2) A possible issue could be the problematic sites are forcing TLS 1.3 which is only supported by R81+, but you are using the R81 release so that is unlikely.

3) A little-known fact: When Categorize HTTPS Sites is enabled, the firewall performs certificate verification similarly to what happens when full HTTPS Inspection is enabled. It is likely that the CA certificate chain for these sites cannot be validated by the firewall (and your curl output seems to suggest this) because it does not have the CA certificate for all CAs in the signing chain utilized by the digital certificates being provided by the problematic sites. Please provide a screenshot of the settings on Manage & Settings...Blades...HTTPS Inspection...SmartDashboard...HTTPS Inspection...HTTPS Validation, as you may have "Untrusted server certificate" checked.

4) Inspect the signing chain for the certificates being presented by the problematic systems, and make sure all entities are included on the "HTTPS Inspection...Trusted CAs" screen in the SmartDashboard. If they are not it may be as simple as clicking the "Install Now" button at the bottom of this screen to update with the latest CAs, as this update does NOT happen automatically and it is possible the problematic CAs are new and simply not in the trusted list for the firewall, but are for the web browsers. You may possibly need to manually import the needed chain certificate instead to complete the signing chain properly. See the following for an example: sk121615: HTTPS Inspection rejects Netflix traffic

5) Unlikely, but it is possible that the problematic sites are using non-RFC compliant HTTPS, which will suddenly break them when Categorize HTTPS Sites is enabled. See here: sk109897: How to use "Categorize HTTPS websites" configuration with non-RFC compliant HTTPS traffic

Most of this was mentioned in my IPS/AV/ABOT Video Class which covers HTTPS Inspection...

Re: Categorize HTTPS Websites

Nandhakumar — Sun, 21 Nov 2021 03:56:37 GMT

Thanks @Timothy_Hall for your response. In our infra, we haven't enabled https inspection. The specific question runs in my mind "how it was working fine for other internet sites except few websites?"

The problematic site is complaint with RFC 5246 (TLS V1.2). Please find attached screenshot for HTTPS Validation where "Untrusted Server Certificate" was not checked.

Also, does firewall need client machine certificate to be in the trusted CA list in order for outbound traffic to work or no need? (Assume here we enable only 'categorize https websites' option)

Re: Categorize HTTPS Websites

the_rock — Sun, 21 Nov 2021 02:55:39 GMT

What do you see in the logs if you just do search for url filtering blade when you try accessing those websites? Can you share example of one of the logs?

Re: Categorize HTTPS Websites

mcatanzaro — Sun, 21 Nov 2021 03:46:40 GMT

Hi,

Your curl output shows as such as you didn't use the '-k' switch for https (ignores warnings/errors in the case you didn't provide the full certificate chain to curl. Make sure you trust the site...).

Do you have an example of one of the domains that is giving you issues?

Tim listed a good number of possibilities of what the issue could be but it would help to have an example site.

Re: Categorize HTTPS Websites

Timothy_Hall — Mon, 22 Nov 2021 15:20:51 GMT

Yes I realize that you don't have HTTPS Inspection enabled, but when Categorize HTTPS Sites is enabled it performs certificate verification and that is probably what is failing. The firewall should not need to have the Client cert CA. Either the traffic to those few websites is not RFC compliant or there is something different about the certificates those sites are using, and I've already mentioned how to investigate these possibilities.

Re: Categorize HTTPS Websites

_Val_ — Sun, 21 Nov 2021 15:15:04 GMT

So that is what I expected. GW cannot validate certificates properly

Re: Categorize HTTPS Websites

genisis__ — Sun, 21 Nov 2021 21:45:32 GMT

Believe it or not I have a similar issue access, of all sites "bbc.co.uk". We are running R80.40 with JHFA118 and in VSX mode.

I've enabled https inspect on a test VS. cnn.com works fine and bbc.co.uk fails everytime with https enabled.

I raise a TAC case, months ago and TAC have not provide any resolution.

What I do know is the Global Root CA used is imported into the repository (by default), the intermediates are not there (yes did try to import this as well and TAC were on a zoom with me) but that should not really matter.

The only way I could get this working was to import the actually 'bbc.co.uk' certificate, which is totally wrong (TAC have also seen this).

For me using https inspection is completely useless with Checkpoint for two reason:

- When we have a site that does not work, the evidence so far indicates that Checkpoint cannot resolve it within a reasonable timeframe.

- The resource requirements to use https inspection is just not financially viable, its better to use a cheaper competitor that does this at a fraction of the cost and with dedicate inbuild https inspection module without the hefty price tag.

Sorry this sounds like a rant, but unfortunately this has been my experience.

also one odd thing that TAC mentioned which I've challenged:

Apparently VS0 needs access to the internet for OSCP access, despite the fact I see no traffic from VS0 even attempting this.