topic Re: Google Searches Odd Behavior in Firewall and Security Management

Google Searches Odd Behavior

jberg712 — Wed, 14 Aug 2024 15:55:45 GMT

I wanted to reach out to see if anyone has been experiencing similar issues. We've had several users indicate to us that when they do Google searches and attempt to click on a link, it just hangs and spins. You can click the address bar and hit enter a couple of times or re-click the link a few times and it finally goes to the link. I can take the system out from behind the firewall and everything works seamlessly. I've even put a troubleshooting rule to allow all outbound for myself to test and I still experience the same issue. Strangely enough, if I use firefox, I don't experience the issue. It's in Edge and Chrome which led me to believe a recent update occurred to created a problem. But again, when I take the system out from behind the firewall, everything works fine. So I don't know if there's a component in Chrome/Edge that CheckPoint is struggling with parsing or what. Again, even adding a rule that allows all outbound access doesn't change the behavior. Has anyone else come across this behavior?

Re: Google Searches Odd Behavior

the_rock — Wed, 14 Aug 2024 15:57:40 GMT

Never had that issue myself, even in the lab with ssl inspection on. Do you use ssl inspection? Regardless, when did the issue happen? Is it affecting all users?

Andy

Re: Google Searches Odd Behavior

AdiGH — Wed, 14 Aug 2024 16:05:58 GMT

Are you using Harmony Browse or Harmony Endpoint products? or only the firewall?

Re: Google Searches Odd Behavior

jberg712 — Wed, 14 Aug 2024 17:07:28 GMT

We do use SSL Inspection. It appears to be affecting all users behind the gateway. I know some services with Google are inspected and some are not.

Re: Google Searches Odd Behavior

jberg712 — Wed, 14 Aug 2024 17:09:12 GMT

We are not using Harmony Endpoint or Browse. It's only Secure Gateway with Firewall, App Cntrl, URL filtering, HTTPS Inspection, along with threat protection like IPS, Antibot, Antivirus.

Re: Google Searches Odd Behavior

Lesley — Wed, 14 Aug 2024 18:06:17 GMT

I think this one is the issue:

https://support.checkpoint.com/results/sk/sk111754

Re: Google Searches Odd Behavior

the_rock — Wed, 14 Aug 2024 19:29:40 GMT

Definitely could be related...

Andy

Re: Google Searches Odd Behavior

the_rock — Wed, 14 Aug 2024 19:38:21 GMT

Is it same behavior no matter what browser they use? I attached a doc with some screenshots of how I have this set up in the lab and works fine.

Andy

Re: Google Searches Odd Behavior

PhoneBoy — Thu, 15 Aug 2024 17:13:01 GMT

You posted this issue in a space for Harmony products, which are Endpoint based.
This sounds like a gateway issue, so I'm moving this thread to the correct area.

Re: Google Searches Odd Behavior

PhoneBoy — Thu, 15 Aug 2024 17:15:05 GMT

Please provide version/JHF of gateway as well as any logs that occur during the time this problem is occuring.
I would explicitly block QUIC in your access policy if you have not already as this is not supported for HTTPS Inspection until R82.

Re: Google Searches Odd Behavior

jberg712 — Tue, 20 Aug 2024 14:17:39 GMT

I actually did some more digging into this. (We have the QUIC protocol disabled and have for sometime now across the board). What I noticed is that there is some odd correlation between Chrome/Edge and the HTTP2 protocol. When I disable HTTP2 by using the flag on the shortcut --disable-http2, traffic to sites works tremendously better. The logs are indicating when I go to a site like google and do a search, I'm matched properly to the correct rule, however, at some point a Redirect log is generated and the redirect does not match properly and it's being picked up on the a compliance rule. The best example I have so far of this is a user in marketing department who uses facebook to update our facebook page. I'll attach the screenshots.

I've dug around and all I can see when it comes to HTTP2 is to disable strict hold on SK116022 along with SK180257 and SK180673. I'm afraid to disable inspecting of HTTP2 with fear that leads to things being vulnerable during web browsing for the end user. Disabling strict hold didn't help the situation.

I have a TAC ticket open about this. Are there any recommendations/best practices when it comes to HTTP2 protocol? It's been out for nearly 10 years now so I can't imagine why there would still be issues with it. Unless Chrome made some update in the latest releases that did something funky with it. I can't find good info on release notes for Chrome/Edge releases. I say that because in one scenario I put on Firefox for another user and his works without issue unlike Chrome/Edge. It's like a perfect mixture of chaos between Chrome, CheckPoint, and HTTP2.

We are currently on R81.20 Take 76

Re: Google Searches Odd Behavior

PhoneBoy — Tue, 20 Aug 2024 15:18:09 GMT

We support HTTP/2 for HTTPS Inspection as of R81 (though I believe it requires USFW support).
What hardware is this on?
HTTP/3 will be supported for HTTPS Inspection in R82.

Re: Google Searches Odd Behavior

jberg712 — Tue, 20 Aug 2024 15:20:28 GMT

It's on a 6600 SGW.

Re: Google Searches Odd Behavior

Lesley — Tue, 20 Aug 2024 17:45:27 GMT

Hi,

Please check bug:

PRJ-51049,
PMTR-97496

Security Gateway

In some scenarios, websites that use HTTP2 protocol do not load properly.

This is solved in R81.20 take 79 (released yesterday for R81.20 and today for R81.10

Re: Google Searches Odd Behavior

jberg712 — Tue, 20 Aug 2024 18:16:35 GMT

Thank you for pointing and finding this out Lesley. I'm going to prepare to update my management and gateways with this to see if this resolves the issue.

Re: Google Searches Odd Behavior

jberg712 — Tue, 20 Aug 2024 18:19:52 GMT

@PhoneBoy Question regarding USFW and KSFW. I believe in the past we've always been recommended to use KSFW. I believe when we were running R81.10 on our 6600, we did end up swapping to KSFW because of strange performance issues. It may have reverted back to USFW when we did the upgrade to R81.20. What is usually recommended for overall performance? Should we stay with USFW or consider moving to KSFW?

Re: Google Searches Odd Behavior

Lesley — Tue, 20 Aug 2024 18:38:16 GMT

This depends. Sharing below output could help to answer the question.

Best Practices

Use the factors listed below to select the best CoreXL Firewall mode for your Security Gateway - User Space (USFW) or Kernel Space (KSFW):

Factor Testing command Recommended
Firewall
mode

80% or more of the traffic undergoes the Fast / Accelerated path fwaccel stats -s KSFW

70% or more of the traffic undergoes the Firewall / Slow path fwaccel stats -s KSFW

30% or more of the traffic undergoes the PXL / Medium path fwaccel stats -s USFW

Security Gateway is configured with more CoreXL SND instances than CoreXL Firewall instances,
or when CoreXL SND instances are the bottleneck fw ctl affinity -l -r KSFW

Security Gateway is configured with more than 38 CoreXL Firewall instances fw ctl affinity -l -r USFW

Re: Google Searches Odd Behavior

jberg712 — Tue, 20 Aug 2024 18:50:12 GMT

@Lesley I was looking at that as well Lesley. The part i'm having trouble understanding is the 'fw ctl affinity' results.

Here are the fwaccel stats:

fwaccel stats -s
Accelerated conns/Total conns : 18/14309 (0%)
LightSpeed conns/Total conns : 0/14309 (0%)
Accelerated pkts/Total pkts : 10138306524/10716219296 (94%)
LightSpeed pkts/Total pkts : 0/10716219296 (0%)
F2Fed pkts/Total pkts : 577912772/10716219296 (5%)
F2V pkts/Total pkts : 157378648/10716219296 (1%)
CPASXL pkts/Total pkts : 7960499954/10716219296 (74%)
PSLXL pkts/Total pkts : 2176883828/10716219296 (20%)
CPAS pipeline pkts/Total pkts : 0/10716219296 (0%)
PSL pipeline pkts/Total pkts : 0/10716219296 (0%)
QOS inbound pkts/Total pkts : 0/10716219296 (0%)
QOS outbound pkts/Total pkts : 0/10716219296 (0%)
Corrected pkts/Total pkts : 0/10716219296 (0%)

The 'fw ctl affinity' says this:

fw ctl affinity -l -r
CPU 0:
CPU 1:
CPU 2: fw_3 (active)
cprid lpd mta_monitor cp_file_convertd usrchkd mpdaemon fwd wsdnsd rad cserver rtmd pepd watermark_cp_file_convertd in.aftpd in.msd scrubd in.ahclientd scanengine_b in.asessiond scrub_cp_file_convertd core_uploader pdpd in.emaild.mta vpnd in.acapd cprid cpd msgd
CPU 3: fw_2 (active)
cprid lpd mta_monitor cp_file_convertd usrchkd mpdaemon fwd wsdnsd rad cserver rtmd pepd watermark_cp_file_convertd in.aftpd in.msd scrubd in.ahclientd scanengine_b in.asessiond scrub_cp_file_convertd core_uploader pdpd in.emaild.mta vpnd in.acapd cprid cpd msgd
CPU 4: fw_1 (active)
cprid lpd mta_monitor cp_file_convertd usrchkd mpdaemon fwd wsdnsd rad cserver rtmd pepd watermark_cp_file_convertd in.aftpd in.msd scrubd in.ahclientd scanengine_b in.asessiond scrub_cp_file_convertd core_uploader pdpd in.emaild.mta vpnd in.acapd cprid cpd msgd
CPU 5: fw_0 (active)
cprid lpd mta_monitor cp_file_convertd usrchkd mpdaemon fwd wsdnsd rad cserver rtmd pepd watermark_cp_file_convertd in.aftpd in.msd scrubd in.ahclientd scanengine_b in.asessiond scrub_cp_file_convertd core_uploader pdpd in.emaild.mta vpnd in.acapd cprid cpd msgd
All:
Interface eth1: has multi queue enabled
Interface eth5: has multi queue enabled
Interface eth1-01: has multi queue enabled
Interface eth1-02: has multi queue enabled
Interface Mgmt: has multi queue enabled

I'm not sure how to read how many CoreXL instances vs CoreXL SND instances from these results. Are you able to tell?

Re: Google Searches Odd Behavior

Lesley — Tue, 20 Aug 2024 18:53:56 GMT

Looks like 2 SND and the res FWK's to be sure verify with cpview -> cpu

You will see either CoreXL_SND or OTHER. And CoreXL_FW

Re: Google Searches Odd Behavior

jberg712 — Tue, 20 Aug 2024 18:58:58 GMT

CPview indicates 2 CPUs for CoreXL_SND and 4 CPUs for CoreXL_FW

Factor	Testing command	Recommended Firewall mode
80% or more of the traffic undergoes the Fast / Accelerated path	`fwaccel stats -s`	KSFW
70% or more of the traffic undergoes the Firewall / Slow path	`fwaccel stats -s`	KSFW
30% or more of the traffic undergoes the PXL / Medium path	`fwaccel stats -s`	USFW
Security Gateway is configured with more CoreXL SND instances than CoreXL Firewall instances, or when CoreXL SND instances are the bottleneck	`fw ctl affinity -l -r`	KSFW
Security Gateway is configured with more than 38 CoreXL Firewall instances	`fw ctl affinity -l -r`	USFW