Traffic Slowdowns with Web Categorization?

[Eric_Boughton]

I'm looking to figure out if turning the web categorization setting from background to hold is going to cause any delay in traffic. 
When we first setup the our new checkpoint cluster the setting was on to hold. I was hearing reports of users with slow traffic. One of the settings that helped was changing the categorization from hold to background. We gained about a second or 2 for load times. We are now 7 months into using the new devices and I'd like to try changing this back to hold. One note is we are university with students living on campus and try to not restrict what they are able to do. We do want to protect them from malware/bots etc. 

Will turning Web Categorization to hold cause any additional delay? 

Is there any way to change this setting for a network? ( I don't think so)

Are the Anti-Bot settings for categorization different then the Web Categorization ones? I'm referring to the settings under the Threat Prevention Engine Settings.  

 Thank you.

[Timothy_Hall]

To minimize the performance effect of "Hold" see the following from my book:

Special Case: DNS and the rad Daemon

The Resource Advisor Daemon ( rad ) is a key process for many of the commonly used blades listed in the table above. The rad process handles interaction between the firewall and the Check Point ThreatCloud for dynamic lookups of content such as URLs; as such it needs reliable access to the Internet and timely DNS responses to avoid potential delays of user traffic. To ensure that all DNS servers defined in Gaia are reachable and delivering timely responses (which rad actively depends on), run this quick test:

1. On the firewall from expert mode, run cat /etc/resolv.conf and note the DNS servers listed there. For our example the listed DNS servers will be 8.8.8.8 and 4.2.3.2.
2. Now make sure that all DNS servers listed are reachable and responding promptly with the nslookup command like this:

   See that? Looks like the second DNS server’s IP address has a typo (it should be 4.2.2.2), get that fixed! Make sure all DNS servers in the firewall’s list are correct and responding promptly, or DNS resolution delays experienced by the rad daemon could pass through to user sessions.

   As mentioned numerous times throughout this book, it is very important to have the most recent GA Jumbo Hotfix Accumulator loaded on your firewall to obtain the latest performance fixes, and the critical rad process is no exception. Still not convinced? Well then feel free to check out these performance-related problems with the rad daemon that can be easily avoided by loading up the latest GA Jumbo HFA:

• sk103422: Resource Advisor (RAD) does not reuse connections (opens new connection for each request)
• sk106170: Random issues with HTTP web browsing - traffic latency increases, and at some point web browsing stops working
• sk111578: RAD daemon might shutdown due to SIGPIPE signal, which causes functionality issues with various Software Blades that rely on this daemon

   If the Website Categorization Mode has been set to Hold, and an unacceptable level of latency is encountered by users subject to the features that depend on the rad daemon (and you’ve tested the firewall’s DNS connectivity as detailed above), additional statistics can be gathered from the rad process for further troubleshooting. The command is rad_admin stats on (urlf|appi|malware|av) . Then to view ThreatCloud interaction statistics after having enabled them, run cpview and select Advanced...RAD.

   One other situation involving the rad daemon to watch out for: if URL Filtering is enabled and the rad daemon seems to be constantly consuming large amounts of CPU time, it is possible that the URL Filtering cache is constantly overflowing. This cache is sized at a maximum of 20,000 entries by default (which is usually sufficient for up to 1,000 users) and if it overflows, the entire cache will be cleared thus causing a big flurry of URL Filtering lookups to the Check Point ThreatCloud by the rad daemon as it repopulates the cache. If the cache is constantly overflowing this can lead to persistently high CPU usage by the rad daemon, and cause noticeable user web traffic delays if the Website Categorization Mode is set to “Hold”. The current URL Filtering cache utilization can be checked with this command: fw tab -t urlf_cache_tbl -s

   The URL Filtering cache size should not be increased from the default of 20,000 unless you are certain this overflow situation is occurring in your firewall. To modify the URL Filtering cache size, consult: sk90422: How to modify URL Filtering cache size?.

--
"IPS Immersion Training" Self-paced Video Class
Now Available at http://www.maxpowerfirewalls.com

[Eric_Boughton]

I've modified the URL cache setting after confirming our DNS settings are correct. We will be giving the Hold category a try next week. I'm still wondering the difference between the web categorization setting and the threat engine categorization. Does the threat engine apply to all traffic or only suspect traffic? 

Web Control - SmartConsole R80.10 Help 

vs

Web Service - Custom - SmartConsole R80.10 Help 

[PhoneBoy]

I believe all of certain kinds of traffic, since it's not always immediately clear traffic is malicious or not.

DNS lookups is the first thing that comes to mind.