awilk99fbe18d-5cf1-4543-9936-535b8747c024

HTTPS inspection real life examples and caveats in R77.30

Discussion created by awilk99fbe18d-5cf1-4543-9936-535b8747c024 on Apr 26, 2018
Latest reply on Apr 26, 2018 by Vladimir Yakovlev

Hi,

I have done some testing on behalf of a customer who has had issues with HTTPS inspection. Based on this I thought it would make sense to write up my findings for my colleagues in our CP support team as well as submit them for everyone interested to confirm or disprove.

 

The tests performed are meant to show the real-life effects and benefits of using HTTPS inspection with and without (CP or a 3rd party system in a DMZ as) a proxy as well as (not) enabling probe bypass. The firewall used was R77.30 open server with a relatively recent JHF, I believe it was 301.

 

Lets get started with it…

 

1. Using the CP as a proxy or having a proxy in a DMZ

This might be new for some – it certainly surprised me - but logging is much better when using a proxy – either “behind” the CP e.g. in a DMZ or using the CP itself as a proxy. The below tests were done with a website www.uvex-safety.co.uk

 

This is the situation when CP is only seeing the SSL connection without a proxy request:

 

Note that some relevant but far from exact information is in the right-hand-side column which is “Resource”. The “Destination” on the LHS column is rather distracting and near irrelevant at best.

For the non-proxied request logging you get the result of reverse lookup in the Destination column (see above screenshot) while the resource column is containing just the MAIN CN only (see screenshot below) but not the actual website that the user went to – which is one of dozens of alternate names in the certificate.

 

 Screenshot2

 

Compare this with the situation when the CP is used as a proxy:

 

Concluding: when going to a website like the www.uvex-safety.co.uk website with subject alternate names, you’ll only find www.uvex-safety.co.uk in the CP logs when you use the CP as a proxy – the destination in this case is the CP itself and the resource column contains the interesting data.

 

Another good reason for considering the use of “a” proxy (may it be the CP or not), is that it fixes some HTTPS inspection incompatibilities: See sk92888: “If a proxy server is used in the organization, place the Security Gateway between the end user and the proxy (instead of placing it between the proxy and the Internet). This will allow Bypass rules based on category to work. “

 

I have found that the blocking works regardless when using www.uvex-safety.co.uk as the URL in URL filtering but as per that SK you’ll need a proxy request to arrive at or traverse the firewall for category actions to work.

 

2. Using the CP with probe bypass AKA enhanced_ssl_inspection

 

Please note that other forum articles seem to indicate that there will be enhancements to probe bypass for later versions of R80.10. See this thread: https://community.checkpoint.com/message/18249-re-is-there-any-workaround-for-sni-https-traffic-when-enabled-the-https-p…  

 

Enhanced_ssl_inspection has a severe limitation so the decision to turn on this parameter should probably not be taken lightly. See the section from sk104717 at the very bottom of this post. To put it into simple words the default setting enhanced_ssl_inspection 0 can or will break

  • Non-Browser Applications
  • Client certs
  • First connections even if bypassed

 

Which is why CP probably came up with the enhanced_ssl_inspection aka probe bypass…. Now lets look at this enhanced mechanism, surely “enhanced” means “better” and the single limitation seems innocuous enough to just turn on the enhanced mode/probe bypass:

  • HTTPS Inspection will not work for sites that require SNI extension in the SSL "Client hello" packet.

 

It sounds innocuous because most people don’t come across SNI frequently unless they host many websites themselves. SNI is being used by big webhosters that host multiple webpages on one IP and can’t or don’t want to put all the domains hosted on that one IP into one certificate  – think amazon AWS and the like. As per https://blogs.akamai.com/2017/03/reaching-toward-universal-tls-sni.html , more than 30% of the top 100.000 websites are (on HTTPS) SNI only.

 

This translates into two things:

  • All HTTPS accesses that traverse the firewall towards an IP address that is hosted at a big webhoster with HTTPS need bypassing BY IP
  • By the nature of bypassing access to a website that requires SNI (probably because a webhoster has multiple webpages on the IP) it will mean that you’ll bypass not just the very website that you’ve resolved to an IP but also an unknown number of other ones

 

The first point means this must be a difficult scenario to manage and the second means it is insecure on top of this.

 

Solution:

It seems that both using a proxy request to or through the CP (see the top section of this post) or disabling enhanced_ssl_inspection allow access to the site. This time the tests were done against another webpage but with the same firewall as per the top of this post. This time the URL is prescription.uvex-safety.co.uk. As opposed to the www.uvex-safety.co.uk website, this is not using subject alternative names but SNI instead probably because it is hosted with lots of other URLs on an Amazon IP.

 

At the bottom of this post you see screenshots of the behaviour when probe bypass is enabled in these two screenshots:

 

 

Client side:

 

On the internal side you only ever see syn requests. The firewall learns the destination IP the connection is going to go to and sends its own SYN and SSL negotiation on the server side:  

 

Server side:

 

On the external side you see SSLv3 Client Hello going out, followed by a reset from the server as the firewall couldn’t learn the hostname at this point yet as the client connection has “only” proceeded to the SYN. While that SYN contains the destination IP, it doesn’t contain the hostname, hence no chance to do SNI on the outside and indicate to the webserver what page the client wants to request.

 

The next screenshot shows the behaviour when probe bypass is disabled: There is an SNI packet that actually gets sent across.

 

 

I’ve repeated the tests with proxy and probe bypass enabled as well and they were successful so this is another reason to consider proxy requests over straight tunnels to the HTTPS server.

 

PS: The most bizarre thing for me was the fact that you would also get connectivity when you DON’T have any bypasses configured as per the APCL example below – when I had enable even non-matching bypasses, access wouldn’t work.

I guess the CP didn't need to wait for the server to provide any information as to whether the first 2 policies matched and just went ahead sending the traffic on as there were no URL or category based bypasses so basically it didn't need doing the probe bypass, even though it was technically enabled.

  

PPS: When you test this yourself, note that there seems to be some cache on the CP so when you transition from off to on, the sites don’t immediately fail, you’ll have to restart the firewall or try a request you’ve not done before.

  

 

Please let me know your observations / explanations / comments / questions.

Cheers

Albert

 

 

 

 

 

Improvements in HTTPS Inspection Bypass mechanism - Probe Bypass

Important Note: Probe Bypass should not be used if there is a proxy between the Security Gateway and the Internet.

Limitations of HTTPS Inspection Bypass Mechanism without Probe Bypass:

  • Every first connection to a site is inspected even if it should have been bypassed according to the policy.
  • Non-Browser Applications connections are dropped when HTTPS Inspection is enabled (even if bypass is configured).
  • Client certificate connections are dropped when HTTPS Inspection is enabled (even if bypass is configured).

Improvements introduced by Probe Bypass:

  • Bypass mechanism was improved to better reflect policy and resolve the above limitations:
    • Stop the inspection of the first connection to bypassed sites.
    • Allow bypass of Non-Browser Applications connections.
    • Allow Bypass of connections to servers that require client certificate.
  • New probing mechanism eliminates the need to inspect the first connection to an IP address unless it is required by the policy.

Limitations of HTTPS Inspection Bypass Mechanism with enabled Probe Bypass:

  • HTTPS Inspection will not work for sites that require SNI extension in the SSL "Client hello" packet.

Status of Improved HTTPS Inspection Bypass feature (Probe Bypass) is controlled by the value of kernel parameter enhanced_ssl_inspection:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Outcomes