topic Re: RAD configuration "number_of_threads" in Firewall and Security Management

RAD configuration "number_of_threads"

Wolfgang — Fri, 27 Dec 2024 15:29:12 GMT

Following sk182137 - "Internal system error in HTTPS Inspection due to categorization service error (Trap error)" the setting of "number_of_threads" should be changed from (0) to (10).

Please can someone confirm changing these settings ?

We are investigating some issues with high CPU for RAD procces and problematic Internet access. I've found this value is always set to (0) after installation. Does the RAD process runs multithread after changing ? I thought RAD is already multithreaded with the newer releases ?

our log is flooded with errors like this:

Error occurred while accessing:XXXXXXXXXXXXXXXXXXXXXX.profile.iah50-c2.cloudfront.net
RAD request exceeded maximum handing time, check /opt/CPsuite-R81.20/fw1///log/rad_events/Errors/flow_23307_643249

timeout values for URLF and AMWS are already changed to 7200 in rad_conf.C

Re: RAD configuration "number_of_threads"

AlekzNet — Fri, 27 Dec 2024 21:13:26 GMT

I have a case opened for various RAD issues since October last year. This is one of them. This "number_of_threads" variable was changed by the assigned engineer and R&D multiple times from 0 to double the cores. The last recommended setting is "1".

I would suggest to open a case (if you have a contract) - hopefully, the RAD problems will get more attention.

This is our current $FWDIR/conf/rad_conf.C:

(
:urlfs_service_check_seconds (7200)
:amws_service_check_seconds (7200)
:cpu_cores_as_number_of_threads (false)
:number_of_threads (1)
:threads_to_cores_ratio (0.334)
:minimal_resources_usage_ratio (0.2)
:number_of_threads_fast_response (4)
:number_of_threads_slow_response (5)
:number_of_threads_zph_response (0)
:number_of_threads_update (0)
:queue_max_capacity (4000)
:debug_traffic (false)
:use_dns_cache (true)
:dns_cache_timeout_sec (2)
:use_ssl_cache (true)
:cert_file_name ("ca-bundle.crt")
:cert_type ("CRT")
:ssl_version ("TLSv1_0")
:ciphers ("TLSv1")
:autodebug (false)
:timeout_events (false)
:normal_flow_events (false)
:log_timeouts (false)
:log_errors (true)
:number_of_reports (512)
:max_repository_multiplier (20)
:flow_timeout (6)
:excessive_flow_timeout (120)
:transfer_timeout_sec (100)
:max_flows (3500)
:max_pc_in_reply (0)
:max_reqs_per_conn_pool (50)
:retry_mechanism_on (true)
:max_retries (25)
:retry_peroid_mins (15)
:happy_eyeballs_timeout (200)
:large_scale_min_cpus (100)
:large_scale_max_threads (70)
:max_threads (32)
)

Re: RAD configuration "number_of_threads"

AlekzNet — Fri, 27 Dec 2024 22:06:29 GMT

Also, you might want to increase the cache sizes in GUIDBEdit for:

Other > rad_services > [malware_rad_service_0 ; dns_rad_service_0 ; urlf_rad_service_0] up to cache_max_hash_size

Re: RAD configuration "number_of_threads"

AlekzNet — Fri, 27 Dec 2024 22:25:31 GMT

Oh yes, and set these to "false":

:debug_traffic (false)
:autodebug (false)

Re: RAD configuration "number_of_threads"

AlekzNet — Fri, 27 Dec 2024 23:02:29 GMT

Re: RAD configuration "number_of_threads"

the_rock — Sat, 28 Dec 2024 00:14:17 GMT

I see all @AlekzNet is saying. Personally, I would still confirm all this with TAC.

Re: RAD configuration "number_of_threads"

Lesley — Mon, 30 Dec 2024 18:47:02 GMT

Any changes to $FWDIR/conf/rad_conf.C: I would recommend to consult with TAC.

Every setup is different and require different values.

Re: RAD configuration "number_of_threads"

AlekzNet — Tue, 14 Jan 2025 20:58:39 GMT

Looks like RAD does not cache the results and queries http://cws.checkpoint.com for each and every passing URL, even if they are constantly repeated. I already proved that to R&D twice, but, apparently, they are still not convinced...

Re: RAD configuration "number_of_threads"

AlekzNet — Thu, 16 Jan 2025 13:14:07 GMT

Some other thoughts. RAD uses plain HTTP to send all domain names addressed from within the company to CheckPoint in, literally, plain text in the URL. What does not look good from the privacy and confidentiality perspective.

It also does not look good from the performance perspective, because instead of creating one HTTPS connection, and sending names/URLs inside the channel, RAD uses curl on the per name basis.

Obviously, the absence of caching and using CURL instead of a single HTTPS channel means very easy and simple programming, but at what expense?

Re: RAD configuration "number_of_threads"

Timothy_Hall — Thu, 16 Jan 2025 18:16:46 GMT

Yes the RAD daemon has had a long and checkered past; when it starts to struggle Internet surfing performance begins to suffer in noticeable ways. It got much better in R80.40 when they redid the daemon and multi-threaded it. The requests sent by RAD are indeed sent in the clear but that can be changed. If you are seeing the RAD daemon request the same site over and over again, the URL cache may be thrashing and need to be increased. All these topics are covered in by Gateway Performance Optimization Course; the relevant pages for RAD are below:

Re: RAD configuration "number_of_threads"

the_rock — Thu, 16 Jan 2025 18:32:12 GMT

@Timothy_Hall I was actually going through your book (2nd edition from 2018 is one I got on Amazon in 2023), but dont see that section. Maybe it was not listed in that release, but I totally get the point you made, had that issue happen with customers few times before.

Andy

Re: RAD configuration "number_of_threads"

AlekzNet — Thu, 16 Jan 2025 20:35:00 GMT

> the URL cache may be thrashing and need to be increased.

They are already increased to 250-300k in dbedit:

Other > rad_services > [malware_rad_service_0 ; dns_rad_service_0 ; urlf_rad_service_0] up to cache_max_hash_size

Since every request goes to the CP site, either the cache is not used (a bug) or the cache is not expired/renewed (a bug).

"RAD over HTTPS " is a simple replacement HTTP with HTTPS, not creating a single connection I was talking about, hence the performance is obviously degraded.