RAD configuration "number_of_threads"

[Wolfgang]

Following sk182137 - "Internal system error in HTTPS Inspection due to categorization service error (Trap erro... the setting of "number_of_threads" should be changed from (0) to (10).

Please can someone confirm changing these settings ?

We are investigating some issues with high CPU for RAD procces and problematic Internet access. I've found this value is always set to (0) after installation. Does the RAD process runs multithread after changing ? I thought RAD is already multithreaded with the newer releases ?

our log is flooded with errors like this:

Error occurred while accessing:XXXXXXXXXXXXXXXXXXXXXX.profile.iah50-c2.cloudfront.net
RAD request exceeded maximum handing time, check /opt/CPsuite-R81.20/fw1///log/rad_events/Errors/flow_23307_643249

timeout values for URLF and AMWS are already changed to 7200 in rad_conf.C

[AlekzNet]

I have a case opened for various RAD issues since October last year.  This is one of them. This "number_of_threads" variable was changed by the assigned engineer and R&D multiple times from 0 to double the cores. The last recommended setting is "1".

I would suggest to open a case (if you have a contract) - hopefully, the RAD problems will get more attention.

This is our current $FWDIR/conf/rad_conf.C:

(
:urlfs_service_check_seconds (7200)
:amws_service_check_seconds (7200)
:cpu_cores_as_number_of_threads (false)
:number_of_threads (1)
:threads_to_cores_ratio (0.334)
:minimal_resources_usage_ratio (0.2)
:number_of_threads_fast_response (4)
:number_of_threads_slow_response (5)
:number_of_threads_zph_response (0)
:number_of_threads_update (0)
:queue_max_capacity (4000)
:debug_traffic (false)
:use_dns_cache (true)
:dns_cache_timeout_sec (2)
:use_ssl_cache (true)
:cert_file_name ("ca-bundle.crt")
:cert_type ("CRT")
:ssl_version ("TLSv1_0")
:ciphers ("TLSv1")
:autodebug (false)
:timeout_events (false)
:normal_flow_events (false)
:log_timeouts (false)
:log_errors (true)
:number_of_reports (512)
:max_repository_multiplier (20)
:flow_timeout (6)
:excessive_flow_timeout (120)
:transfer_timeout_sec (100)
:max_flows (3500)
:max_pc_in_reply (0)
:max_reqs_per_conn_pool (50)
:retry_mechanism_on (true)
:max_retries (25)
:retry_peroid_mins (15)
:happy_eyeballs_timeout (200)
:large_scale_min_cpus (100)
:large_scale_max_threads (70)
:max_threads (32)
)

[AlekzNet]

Oh yes, and set these to "false":

:debug_traffic (false)
:autodebug (false)

[AlekzNet]

Looks like RAD does not cache the results and queries  http://cws.checkpoint.com  for each and every passing URL, even if they are constantly repeated. I already proved that to R&D twice, but, apparently, they are still not convinced...

[AlekzNet]

Some other thoughts. RAD uses plain HTTP to send all domain names addressed from within the company to CheckPoint in, literally, plain text in the URL. What does not look good from the privacy and confidentiality perspective.

It also does not look good from the performance perspective, because instead of creating one HTTPS connection, and sending names/URLs inside the channel, RAD uses curl on the per name basis.

Obviously, the absence of caching and using CURL instead of a single HTTPS channel means very easy and simple programming, but at what expense?

[Timothy_Hall]

Yes the RAD daemon has had a long and checkered past; when it starts to struggle Internet surfing performance begins to suffer in noticeable ways.  It got much better in R80.40 when they redid the daemon and multi-threaded it.  The requests sent by RAD are indeed sent in the clear but that can be changed.   If you are seeing the RAD daemon request the same site over and over again, the URL cache may be thrashing and need to be increased.  All these topics are covered in by Gateway Performance Optimization Course; the relevant pages for RAD are below:

[the_rock]

@Timothy_Hall I was actually going through your book (2nd edition from 2018 is one I got on Amazon in 2023), but dont see that section. Maybe it was not listed in that release, but I totally get the point you made, had that issue happen with customers few times before.

Andy

[AlekzNet]

> the URL cache may be thrashing and need to be increased.

They are already increased to 250-300k  in dbedit:

Other > rad_services > [malware_rad_service_0 ; dns_rad_service_0 ; urlf_rad_service_0] up to cache_max_hash_size

Since every request goes to the CP site, either the cache is not used (a bug) or the cache is not expired/renewed (a bug).

"RAD over HTTPS " is a simple replacement HTTP with HTTPS, not creating a single connection I was talking about, hence the performance is obviously  degraded. 

[AlekzNet]

Also, you might want to increase the cache sizes in GUIDBEdit for:

Other > rad_services > [malware_rad_service_0 ; dns_rad_service_0 ; urlf_rad_service_0] up to cache_max_hash_size

[AlekzNet]

Related thread: https://community.checkpoint.com/t5/Security-Gateways/High-CPU/td-p/134517

[the_rock]

I see all @AlekzNet  is saying. Personally, I would still confirm all this with TAC.

[Lesley]

Any changes to $FWDIR/conf/rad_conf.C: I would recommend to consult with TAC.

Every setup is different and require different values.