- CheckMates
- :
- Products
- :
- Quantum
- :
- Security Gateways
- :
- Re: Google Searches Odd Behavior
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Are you a member of CheckMates?
×- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Google Searches Odd Behavior
I wanted to reach out to see if anyone has been experiencing similar issues. We've had several users indicate to us that when they do Google searches and attempt to click on a link, it just hangs and spins. You can click the address bar and hit enter a couple of times or re-click the link a few times and it finally goes to the link. I can take the system out from behind the firewall and everything works seamlessly. I've even put a troubleshooting rule to allow all outbound for myself to test and I still experience the same issue. Strangely enough, if I use firefox, I don't experience the issue. It's in Edge and Chrome which led me to believe a recent update occurred to created a problem. But again, when I take the system out from behind the firewall, everything works fine. So I don't know if there's a component in Chrome/Edge that CheckPoint is struggling with parsing or what. Again, even adding a rule that allows all outbound access doesn't change the behavior. Has anyone else come across this behavior?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Never had that issue myself, even in the lab with ssl inspection on. Do you use ssl inspection? Regardless, when did the issue happen? Is it affecting all users?
Andy
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We do use SSL Inspection. It appears to be affecting all users behind the gateway. I know some services with Google are inspected and some are not.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Is it same behavior no matter what browser they use? I attached a doc with some screenshots of how I have this set up in the lab and works fine.
Andy
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Are you using Harmony Browse or Harmony Endpoint products? or only the firewall?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We are not using Harmony Endpoint or Browse. It's only Secure Gateway with Firewall, App Cntrl, URL filtering, HTTPS Inspection, along with threat protection like IPS, Antibot, Antivirus.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You posted this issue in a space for Harmony products, which are Endpoint based.
This sounds like a gateway issue, so I'm moving this thread to the correct area.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I think this one is the issue:
https://support.checkpoint.com/results/sk/sk111754
If you like this post please give a thumbs up(kudo)! 🙂
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Definitely could be related...
Andy
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Please provide version/JHF of gateway as well as any logs that occur during the time this problem is occuring.
I would explicitly block QUIC in your access policy if you have not already as this is not supported for HTTPS Inspection until R82.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I actually did some more digging into this. (We have the QUIC protocol disabled and have for sometime now across the board). What I noticed is that there is some odd correlation between Chrome/Edge and the HTTP2 protocol. When I disable HTTP2 by using the flag on the shortcut --disable-http2, traffic to sites works tremendously better. The logs are indicating when I go to a site like google and do a search, I'm matched properly to the correct rule, however, at some point a Redirect log is generated and the redirect does not match properly and it's being picked up on the a compliance rule. The best example I have so far of this is a user in marketing department who uses facebook to update our facebook page. I'll attach the screenshots.
I've dug around and all I can see when it comes to HTTP2 is to disable strict hold on SK116022 along with SK180257 and SK180673. I'm afraid to disable inspecting of HTTP2 with fear that leads to things being vulnerable during web browsing for the end user. Disabling strict hold didn't help the situation.
I have a TAC ticket open about this. Are there any recommendations/best practices when it comes to HTTP2 protocol? It's been out for nearly 10 years now so I can't imagine why there would still be issues with it. Unless Chrome made some update in the latest releases that did something funky with it. I can't find good info on release notes for Chrome/Edge releases. I say that because in one scenario I put on Firefox for another user and his works without issue unlike Chrome/Edge. It's like a perfect mixture of chaos between Chrome, CheckPoint, and HTTP2.
We are currently on R81.20 Take 76
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We support HTTP/2 for HTTPS Inspection as of R81 (though I believe it requires USFW support).
What hardware is this on?
HTTP/3 will be supported for HTTPS Inspection in R82.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It's on a 6600 SGW.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@PhoneBoy Question regarding USFW and KSFW. I believe in the past we've always been recommended to use KSFW. I believe when we were running R81.10 on our 6600, we did end up swapping to KSFW because of strange performance issues. It may have reverted back to USFW when we did the upgrade to R81.20. What is usually recommended for overall performance? Should we stay with USFW or consider moving to KSFW?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This depends. Sharing below output could help to answer the question.
Best Practices
Use the factors listed below to select the best CoreXL Firewall mode for your Security Gateway - User Space (USFW) or Kernel Space (KSFW):
Factor Testing command Recommended
Firewall
mode80% or more of the traffic undergoes the Fast / Accelerated path fwaccel stats -s
KSFW 70% or more of the traffic undergoes the Firewall / Slow path fwaccel stats -s
KSFW 30% or more of the traffic undergoes the PXL / Medium path fwaccel stats -s
USFW Security Gateway is configured with more CoreXL SND instances than CoreXL Firewall instances,
or when CoreXL SND instances are the bottleneckfw ctl affinity -l -r
KSFW Security Gateway is configured with more than 38 CoreXL Firewall instances fw ctl affinity -l -r
USFW
If you like this post please give a thumbs up(kudo)! 🙂
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Lesley I was looking at that as well Lesley. The part i'm having trouble understanding is the 'fw ctl affinity' results.
Here are the fwaccel stats:
fwaccel stats -s
Accelerated conns/Total conns : 18/14309 (0%)
LightSpeed conns/Total conns : 0/14309 (0%)
Accelerated pkts/Total pkts : 10138306524/10716219296 (94%)
LightSpeed pkts/Total pkts : 0/10716219296 (0%)
F2Fed pkts/Total pkts : 577912772/10716219296 (5%)
F2V pkts/Total pkts : 157378648/10716219296 (1%)
CPASXL pkts/Total pkts : 7960499954/10716219296 (74%)
PSLXL pkts/Total pkts : 2176883828/10716219296 (20%)
CPAS pipeline pkts/Total pkts : 0/10716219296 (0%)
PSL pipeline pkts/Total pkts : 0/10716219296 (0%)
QOS inbound pkts/Total pkts : 0/10716219296 (0%)
QOS outbound pkts/Total pkts : 0/10716219296 (0%)
Corrected pkts/Total pkts : 0/10716219296 (0%)
The 'fw ctl affinity' says this:
fw ctl affinity -l -r
CPU 0:
CPU 1:
CPU 2: fw_3 (active)
cprid lpd mta_monitor cp_file_convertd usrchkd mpdaemon fwd wsdnsd rad cserver rtmd pepd watermark_cp_file_convertd in.aftpd in.msd scrubd in.ahclientd scanengine_b in.asessiond scrub_cp_file_convertd core_uploader pdpd in.emaild.mta vpnd in.acapd cprid cpd msgd
CPU 3: fw_2 (active)
cprid lpd mta_monitor cp_file_convertd usrchkd mpdaemon fwd wsdnsd rad cserver rtmd pepd watermark_cp_file_convertd in.aftpd in.msd scrubd in.ahclientd scanengine_b in.asessiond scrub_cp_file_convertd core_uploader pdpd in.emaild.mta vpnd in.acapd cprid cpd msgd
CPU 4: fw_1 (active)
cprid lpd mta_monitor cp_file_convertd usrchkd mpdaemon fwd wsdnsd rad cserver rtmd pepd watermark_cp_file_convertd in.aftpd in.msd scrubd in.ahclientd scanengine_b in.asessiond scrub_cp_file_convertd core_uploader pdpd in.emaild.mta vpnd in.acapd cprid cpd msgd
CPU 5: fw_0 (active)
cprid lpd mta_monitor cp_file_convertd usrchkd mpdaemon fwd wsdnsd rad cserver rtmd pepd watermark_cp_file_convertd in.aftpd in.msd scrubd in.ahclientd scanengine_b in.asessiond scrub_cp_file_convertd core_uploader pdpd in.emaild.mta vpnd in.acapd cprid cpd msgd
All:
Interface eth1: has multi queue enabled
Interface eth5: has multi queue enabled
Interface eth1-01: has multi queue enabled
Interface eth1-02: has multi queue enabled
Interface Mgmt: has multi queue enabled
I'm not sure how to read how many CoreXL instances vs CoreXL SND instances from these results. Are you able to tell?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Looks like 2 SND and the res FWK's to be sure verify with cpview -> cpu
You will see either CoreXL_SND or OTHER. And CoreXL_FW
If you like this post please give a thumbs up(kudo)! 🙂
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
CPview indicates 2 CPUs for CoreXL_SND and 4 CPUs for CoreXL_FW
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
From a performance perspective, the table @Lesley provided gives you a good idea when to use USFW.
Note that some features require USFW (e.g. HTTPS Inspection for TLS 1.3 and HTTP/2).
I suspect it will be the default mode in a future version.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Please check bug:
PRJ-51049, |
Security Gateway |
In some scenarios, websites that use HTTP2 protocol do not load properly. |
This is solved in R81.20 take 79 (released yesterday for R81.20 and today for R81.10
If you like this post please give a thumbs up(kudo)! 🙂
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you for pointing and finding this out Lesley. I'm going to prepare to update my management and gateways with this to see if this resolves the issue.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Update on this... I updated to Take 79. Still having the odd behavior in Google searches. I have a TAC case open about this. Since the update though, I see logs indicating the 'URL Filtering - Internal System Error (0)'. I've done some debugging and when I did the debug for APPI and NRB I found an error line stating: [ERROR]: rad_kernel_urlf_request_serialize: string len =4288 bigger than max 4096;
I can't seem to find anything more about this in SK's. Below is a broader snippet of the log during the failure event.
@;3832104.2122417;22Aug2024 8:49:15.448813;[vs_0];[tid_0];[fw4_0];1:{global} appi_rad_uf_cmi_handler_is_enabled_ex: with _urlf_blade_enabled=0 -> *_enabled=1;
@;3832104.2122418;22Aug2024 8:49:15.448815;[vs_0];[tid_0];[fw4_0];1:{referrer} appi_clobs_observer_http_inspect_referrer: md5 is 2100488168;
@;3832104.2122419;22Aug2024 8:49:15.448817;[vs_0];[tid_0];[fw4_0];1:{referrer} [WARNING]: appi_clobs_observer_http_inspect_referrer: transaction's application is NULL;
@;3832104.2122420;22Aug2024 8:49:15.448818;[vs_0];[tid_0];[fw4_0];1:{observer} appi_clobs_observer_notify_exe_rb_event: returned retval 0;
@;3832104.2122421;22Aug2024 8:49:15.448833;[vs_0];[tid_2];[fw4_2];1:{policy} appi_rad_uf_cmi_handler_match_cb_handle_url: call rad_kernel_api_get_cached_resource;
@;3832104.2122422;22Aug2024 8:49:15.448857;[vs_0];[tid_1];[fw4_1];1:{policy} appi_rad_uf_cmi_handler_match_cb_handle_url: call rad_kernel_api_get_cached_resource;
@;3832104.2122423;22Aug2024 8:49:15.448867;[vs_0];[tid_2];[fw4_2];[172.16.16.31:53447 -> 108.177.122.147:443] [ERROR]: rad_kernel_urlf_request_serialize: string len =4288 bigger than max 4096;
@;3832104.2122424;22Aug2024 8:49:15.448871;[vs_0];[tid_2];[fw4_2];1:[172.16.16.31:53447 -> 108.177.122.147:443] {policy} [ERROR]: appi_rad_uf_cmi_handler_match_cb_handle_url: rad_kernel_api_async_get_resource_with_status_code() failed, error: Request flat buffer error;
@;3832104.2122425;22Aug2024 8:49:15.448874;[vs_0];[tid_2];[fw4_2];1:[172.16.16.31:53447 -> 108.177.122.147:443] {policy} [ERROR]: appi_rad_uf_cmi_handler_match_cb: appi_rad_uf_cmi_handler_match_cb_handle_url() failed;
@;3832104.2122426;22Aug2024 8:49:15.448876;[vs_0];[tid_2];[fw4_2];1:{global} appi_global_policy_get_fail_action: fail action is REJECT;
@;3832104.2122427;22Aug2024 8:49:15.448877;[vs_0];[tid_2];[fw4_2];1:{connection} appi_rad_uf_cmi_handler_match_cb: perform fail action [Reject];
@;3832104.2122428;22Aug2024 8:49:15.448890;[vs_0];[tid_1];[fw4_1];[172.16.16.31:53446 -> 108.177.122.147:443] [ERROR]: rad_kernel_urlf_request_serialize: string len =4275 bigger than max 4096;
@;3832104.2122429;22Aug2024 8:49:15.448891;[vs_0];[tid_2];[fw4_2];1:{policy} appi_rad_uf_cmi_handler_match_cb: ended;
@;3832104.2122430;22Aug2024 8:49:15.448894;[vs_0];[tid_1];[fw4_1];1:[172.16.16.31:53446 -> 108.177.122.147:443] {policy} [ERROR]: appi_rad_uf_cmi_handler_match_cb_handle_url: rad_kernel_api_async_get_resource_with_status_code() failed, error: Request flat buffer error;
@;3832104.2122431;22Aug2024 8:49:15.448897;[vs_0];[tid_1];[fw4_1];1:[172.16.16.31:53446 -> 108.177.122.147:443] {policy} [ERROR]: appi_rad_uf_cmi_handler_match_cb: appi_rad_uf_cmi_handler_match_cb_handle_url() failed;
@;3832104.2122432;22Aug2024 8:49:15.448899;[vs_0];[tid_1];[fw4_1];1:{global} appi_global_policy_get_fail_action: fail action is REJECT;
@;3832104.2122433;22Aug2024 8:49:15.448900;[vs_0];[tid_1];[fw4_1];1:{connection} appi_rad_uf_cmi_handler_match_cb: perform fail action [Reject];
@;3832104.2122434;22Aug2024 8:49:15.448912;[vs_0];[tid_1];[fw4_1];1:{policy} appi_rad_uf_cmi_handler_match_cb: ended;
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Is the issue still there if you bypass the following categories in HTTPS inspection?
Google Services and Google Cloud Platform Services
If you like this post please give a thumbs up(kudo)! 🙂
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Give me a moment Lesley and I'll test
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Or try below.
Andy
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Very good point @Lesley
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I bypassed Google services (all of them) and I do not get the error. Everything loads like it's supposed to. So it would indicate something with HTTPS inspection?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
100%, if bypass works.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
URLF database, does it show updated in smart console?
Andy
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Andy, it states it's up-to-date, but the latest version is 141202408011245 (Created on 8/1/2024 7:45 AM) Is that the most updated DB?