Check Point Appliance is convinced its unlikely to be replaced ... a Freudian slip .. but how knows?

[Thomas_Eichelbu]

Hello Check Mates, 

some stranges message appear on this machine, also a bit funny. but was does it mean?
another RAD issue ... 

but most funny to see is Check Points funny naming "(flow reached! consider removing CP_UNLIKELY)"
so they are pretty convinced not be removed in the future?

so far we do not feel any impact or symptoms 
but the logs are filling up quite rapidly.

/var/log/messages

Feb 22 09:47:30 2022 XXXXXXXXXXXkernel: [fw4_0];[10.20.46.82:33146 -> 10.90.214.3:80] [ERROR]: malware_res_rep_rad_query: rad_kernel_malware_request_prepare() failed (flow reached! consider removing CP_UNLIKELY)

Feb 22 09:47:35 2022 XXXXXXXXXXXkernel: [fw4_0];[10.20.46.82:45906 -> 10.90.214.4:80] [ERROR]: rad_kernel_malware_request_prepare: invalid normalized_url with prefix slash. normalized_url is /sdktunnel, len=10

Feb 22 09:47:35 2022 XXXXXXXXXXXkernel: [fw4_0];[10.20.46.82:45906 -> 10.90.214.4:80] [ERROR]: malware_res_rep_rad_query: rad_kernel_malware_request_prepare() failed (flow reached! consider removing CP_UNLIKELY)

Feb 22 09:47:46 2022 XXXXXXXXXXXkernel: [fw4_0];[10.20.46.82:33282 -> 10.90.214.3:80] [ERROR]: rad_kernel_malware_request_prepare: invalid normalized_url with prefix slash. normalized_url is /sdktunnel, len=10

Feb 22 09:47:46 2022 XXXXXXXXXXXkernel: [fw4_0];[10.20.46.82:33282 -> 10.90.214.3:80] [ERROR]: malware_res_rep_rad_query: rad_kernel_malware_request_prepare() failed (flow reached! consider removing CP_UNLIKELY)

has anybody seen this before and ha some luck in investigating it?

perhaps its nothing serious.

best regards
Thomas

[_Val_]

It may seem funny to you, but you do have rad issues, and I would sincerely recommend opening a TAC case at once. 

[Thomas_Eichelbu]

Well, not so sure ...
since 7 days i got only about 15 RAD related messages in Smartlog on this specific machine.
they are all like:

"Failed to fetch Check Point resources. Timeout was reached, check /opt/CPsuite-R81/fw1/log/rad_events/Errors/flow_16976_651350 For more details"

ok the uptime is only 2h ... 
i never found any good explanation what this values are all about?
when is it bad, what is normal, when is a threashold reached?

all the cache sizes in this enviroment for APPI/URL and AV/ABot have been increased already ...
still RAD remains a mistery. (at least for me)

[Thomas_Eichelbu]

Well, 

i see only internal clients in this logs, never any external IP´s 
perhaps RAD has issues with categorizing internal resources?

[Diego_dg]

We are seeing the same messages, but in our case related with DNS traffic to 8.8.8.8, did you reach any conclussion about this messages?

[Thomas_Eichelbu]

Hello. 

well i still see them.
R81 + Take 58

messages.9:Jun 15 07:53:37 2022 XXXXXXXXXXX kernel: [fw4_7];[192.168.200.197:44278 -> 8.8.8.8:53] [ERROR]: malware_res_rep_classify_ex: invalid params: _host ffffc901e09618b8, _host_len 0, _conn_data ffff880418aaabb8, _action ffff880418aaac9c (flow reached! consider removing CP_UNLIKELY)

messages.9:Jun 15 07:53:40 2022 XXXXXXXXXXX kernel: [fw4_25];[192.168.200.197:42339 -> 8.8.8.8:53] [ERROR]: malware_res_rep_classify_ex: invalid params: _host ffffc90041a8d8b8, _host_len 0, _conn_data ffff88043c676bb8, _action ffff88043c676c9c (flow reached! consider removing CP_UNLIKELY

also to Google ...
iam no really aware of any impact due this RAD issues ...

but this logs are not listed here:

/opt/CPsuite-R81/fw1/log/rad_events/

so since nobody really complains ... i hope its still informational ... perhaps i will create a TAC Case to get some more information about it ...

[the_rock]

I agree with you, those messages are a bit funny, to me anyway : - ). But, on a serious note, did you ever end up opening a TAC case for this?

[Thomas_Eichelbu]

I have not made any case yet ...
i have so many other cases running, when all high prio cases are finished i will care about this issue ...
i will publish the TAC´s answer then on Check Mates!

[the_rock]

I hear ya brother, I feel your "pain" : - ). Please let us know how it goes.

[Thomas_Eichelbu]

Hello, 

i have created a TAC case now ... 

at the moment i see the most logs for DNS ...

messages.1:Jun 28 09:19:33 2022 XXXXXXXXXX kernel: [fw4_4];[192.168.192.41:43382 -> 8.8.8.8:53] [ERROR]: malware_res_rep_classify_ex: invalid params: _host ffffc9024db01ab0, _host_len 0, _conn_data ffff8804184cebb8, _action ffff8804184cec9c (flow reached! consider removing CP_UNLIKELY)

or this.
messages.7:Jun 23 14:45:36 2022 XXXXXXXXXX  kernel: [fw4_6];[10.XX.XX.66:44704 -> 10.XX.XX:XX:53] [ERROR]: malware_res_rep_rad_query: rad_kernel_malware_request_prepare() failed (flow reached! consider removing CP_UNLIKELY)

so lets see what we find out here!

[Thomas_Eichelbu]

Hello again, 

short update, CP TAC asked us to change the following:

the RAD magic command, i call it the Swiss Army Knife of RAD.

"ckp_regedit -a SOFTWARE\\CheckPoint\\FW1\\$(cpprod_util CPPROD_GetCurrentVersion FW1) RAD_QUERIES_NUMBER_PER_CONNECTION 400"

i asked what number is the best 40, 400, 4000 or what? TAC said, its trial and error just increase the value 😞
the higher the value, the more memory is sucked up by the FW.

to check with:
grep --color -C 1 RAD_QUERIES_NUMBER_PER_CONNECTION $CPDIR/registry/HKLM_registry.data
:APPIUFEnabled (1)
:RAD_QUERIES_NUMBER_PER_CONNECTION (400)
:IsFwdDebugTurnedOn (0)

to make it running, reboot or "rad_admin stop" and "rad_admin start"

If the issue persists:

Edit $FWDIR/conf/rad_conf.C

:amws_service_check_seconds (1800) -> increase this to 3600. **RND recommends keeping this to 3600 at all times if Anti-Virus and Anti-bot are being used.***

:max_pc_in_reply (0) -> Increase to 100 **Same note as above**


GUIDBEdit > Other > rad_services > malware_rad_service > double current cache size

If in
# cat $FWDIR/conf/rad_scheme.C | grep 8.0 ==
:const ("/Malware/malware/8.0?resource=")

then please change it from 8.0 to 6.0 and let me know if there's a change in the behavior.

If the messages reduce in quantity but still appear sometimes:
Increase to:

$FWDIR/conf/rad_conf.C

:amws_service_check_seconds (7200)
:max_pc_in_reply (200)

my config:
cat $FWDIR/conf/rad_conf.C
(
:urlfs_service_check_seconds (7200)
:amws_service_check_seconds (1800)
:cpu_cores_as_number_of_threads (false)
:number_of_threads (0)
:threads_to_cores_ratio (0.334)
:minimal_resources_usage_ratio (0.2)
:number_of_threads_fast_response (0)
:number_of_threads_slow_response (0)
:queue_max_capacity (2000)
:debug_traffic (false)
:use_dns_cache (true)
:dns_cache_timeout_sec (2)
:use_ssl_cache (true)
:cert_file_name ("ca-bundle.crt")
:cert_type ("CRT")
:ssl_version ("TLSv1_0")
:ciphers ("TLSv1")
:autodebug (true)
:timeout_events (false)
:normal_flow_events (false)
:log_timeouts (false)
:log_errors (true)
:number_of_reports (512)
:max_repository_multiplier (20)
:flow_timeout (6)
:excessive_flow_timeout (120)
:transfer_timeout_sec (15)
:max_flows (1000)
:max_pc_in_reply (0)
:retry_mechanism_on (true)
:max_retries (25)
:retry_peroid_mins (15)

iam sorry to say, it has not worked out so far.
at the moment we focus on DNS settings, as we know the clients we saw in the logs use different DNS Servers then the firewall.
so lets see ...

[Thomas_Eichelbu]

Hello, 

"bypass_reverse_dns_rad_request" is the magic command and must be added to $FWDIR/conf/rad_conf.C

Check Point TAC wrote ... 

bypass_reverse_dns_rad_request is a global parameter, when this is enabled the RAD process will not handle reverse DNS requests(i.e. Have the suffix of .ip6.arpa or .in-addr.arpa).

In order to enable it please follow the procedure below:

1)     Backup the current rad configuration file:
# cp $FWDIR/conf/rad_conf.C $FWDIR/conf/rad_conf.C.BACKUP

2)     Edit the configuration file and make the following changes:
# vi $FWDIR/conf/rad_conf.C

       Add the following line (at the end):
                       :bypass_reverse_dns_rad_request (1)

3)     Exit the file and save the changes
4)     Restart the rad process:
# rad_admin restart

questions remain ...
i have to make this settings on 160 firewalls ... how? 
putting the $FWDIR/conf/rad_conf.C to fwrl.conf ??? 
does it get overwritten during updates?
pro/cons of this setting?

and more important does it help?
i will keep you posted!

[farrugial]

Hi,

We seem to be having a similar issue. Did implementing bypass_reverse_dns_rad_request in $FWDIR/conf/rad_conf.C help resolve the issue in the long-term? Thanks for your assistance.