topic Re: HTTPS Inspection & Categorization on R77.30 in General Topics

HTTPS Inspection & Categorization on R77.30

Bharat_B — Mon, 09 Jul 2018 10:29:01 GMT

Dear all,

I've noticed a strange behavior when enabling HTTPS inspection, would like to confirm if anyone has seen a similar problem?

When HTTPS inspection is enabled, page blocking by categorization takes much longer to apply. For e.g. when accessing betting sites over HTTPS or pages with image galleries, the page loads fine and its contents load. Smartview Tracker shows the page contents as blocked but they are loading fine on browser. After retrying approx 30 seconds to 1 minute later, the page is then correctly blocked and Usercheck page is shown.

Is it normal for categorization to take longer on HTTPS inspection or for pages to load successfully for so long? Is there any way to troubleshoot what is happening to cause this delay?

Without HTTPS inspection, pages are blocked almost instantly. Maybe the first time they load partially (images etc don't load) but immediately after the first refresh they are fully blocked (no Usercheck which is normal). With HTTPS inspection, even after the first few refreshes the pages still load. This has been tested with various sites and different categories and the behavior is same.

Re: HTTPS Inspection & Categorization on R77.30

Timothy_Hall — Mon, 09 Jul 2018 14:11:54 GMT

By default, the "Website Categorization Mode" is set to "Background" which means that initial web requests will be allowed even if categorization has not been obtained & cached yet. If you set it to "Hold" (see attached screenshot) the user will not be able to start loading content until it has been categorized (and possibly blocked). If you go this route make sure that all DNS servers configured in the firewall's Gaia OS config are defined properly and responding quickly, or users may suffer long delays trying to load up a new website whose categorization has not yet been cached by the firewall.

In regards to the delay incurred by HTTPS Inspection itself (as opposed to the categorization process discussed above), this feature does cause a process space "trip" on the firewall in R80.10 and earlier. Techniques to minimize the performance impact of the trip in regards to HTTPS Inspection are covered extensively in Chapter 10 of the second edition of my book, and can be roughly summarized as:

Make sure Gaia is running in 64-bit mode
Use firewall hardware whose processor supports AES-NI
Tune the firewall to ensure sufficient free CPU cycles are available on the Firewall Worker cores for the wstlsd/pkxld processes
Avoid use of "Any" in the HTTPS Inspection Policy
Load the latest GA jumbo hotfix
Beware using APCL Limits in conjunction with HTTPS Inspection (sk70600)

--
Second Edition of my "Max Power" Firewall Book
Now Available at http://www.maxpowerfirewalls.com

Re: HTTPS Inspection & Categorization on R77.30

Bharat_B — Thu, 12 Jul 2018 07:04:22 GMT

Hello Tim, thank you for the answer. To reply to the points:

Make sure Gaia is running in 64-bit mode
- Already in 64-bit mode
Use firewall hardware whose processor supports AES-NI
- Hardware is 5400, doesn't look like it supports AES-NI as per This Community post
Tune the firewall to ensure sufficient free CPU cycles are available on the Firewall Worker cores for the wstlsd/pkxld processes
- Firewall is free enough, running at approx 15% CPU
Avoid use of "Any" in the HTTPS Inspection Policy
- Any in HTTP/HTTPS to inspect after the bypass rules. Is this relevant?
Load the latest GA jumbo hotfix
- Already installed T302
Beware using APCL Limits in conjunction with HTTPS Inspection (sk70600)
- Checked

While the blocking eventually happens, the management is not very confident of the solution and believe some blocked content may be slipping through. The worse part is that the logs show the content as "blocked" but the page is actually loading fine in the background for the first few minutes.

Re: HTTPS Inspection & Categorization on R77.30

Timothy_Hall — Thu, 12 Jul 2018 14:29:34 GMT

So is content still "slipping through" even with website categorization set to Hold? It shouldn't be and anything you are seeing to the contrary could just be cached data in the browser.

The main reason to avoid using "Any" in your HTTPS Inspection Policy is to keep LAN-speed traffic between internal networks from accidentally getting sucked into HTTPS Inspection.

It also sounds like the 5400 with its 2 cores may be a bit underpowered for what you are trying to do. What does the output of the "enabled_blades" and "free -m" commands run on the firewall show?

--
Second Edition of my "Max Power" Firewall Book
Now Available at http://www.maxpowerfirewalls.com