topic Re: VPN performance on a 3800 in Firewall and Security Management

VPN performance on a 3800

dj0Nz — Thu, 20 Jul 2023 08:36:04 GMT

Dear community,

I am currently investigating an issue on a CPSG 3800 cluster running only S2S vpns. Throughput is limited to roundabout 300 Mbps because CPU 0 is contantly at 100% load. Besides normal SND tasks, there is the process show below which is causing the load:

Does any of you had similar issues and a solution?

Cheers,
Michael

Re: VPN performance on a 3800

Tal_Paz-Fridman — Thu, 20 Jul 2023 09:51:10 GMT

A similar issue was resolved in JHFs. Check you have the latest installed. For example from R81.10:

https://sc1.checkpoint.com/documents/Jumbo_HFA/R81.10/R81.10/R81.10-List-of-all-Resolved-Issues.htm

PRJ-42145, PMTR-88118

SecureXL

SNDs may reach 100% CPU utilization and are not released in some Site to Site VPN scenarios.

Re: VPN performance on a 3800

dj0Nz — Thu, 20 Jul 2023 10:21:50 GMT

Thank you for your reply. But take 95 is already installed.

Re: VPN performance on a 3800

Tal_Paz-Fridman — Thu, 20 Jul 2023 10:24:59 GMT

I suggest contacting TAC and referencing the fix in the JHF (PRJ-42145, PMTR-88118) so that they can communicate it with relevant owners in R&D.

Re: VPN performance on a 3800

dj0Nz — Thu, 20 Jul 2023 10:29:02 GMT

In the mean time we kind of "solved" it by setting P2 integrity to SHA1 after reviewing sk73980.
But honestly: This gateway is specified to achieve 2.75 IPSEC performance.
It should reach that with up to date crypto and NOT by using deprecated ciphers.

TAC case is open. 😉

Re: VPN performance on a 3800

Chris_Atkinson — Thu, 20 Jul 2023 11:39:42 GMT

So VPN and no blades other than FW?

Have you tuned / altered your CoreXL config at all?

Re: VPN performance on a 3800

the_rock — Thu, 20 Jul 2023 12:04:43 GMT

Just my personal honest opinion...I would NOT do settings from that sk, because it essentially "solves" speed, but severely impacts security as far as VPN goes.

Re: VPN performance on a 3800

dj0Nz — Thu, 20 Jul 2023 12:19:59 GMT

Yes, FW/VPN only and we altered different aspects of CoreXL including disabling dynamic balancing / multiqueueing with no impact on the main problem. We also checked if KSFW mode makes a difference but as expected, it has not.

From the perf top output above I assume, that this is no issue that can be solved on CoreXL level. Looks like a software bug to me.

Re: VPN performance on a 3800

Ruan_Kotze — Thu, 20 Jul 2023 12:24:05 GMT

I've had one case where a customer with around 80 incoming tunnels changed the community from "tunnel per gateway pair" to "tunnel per host pair" and that immediately caused the appliance to become unresponsive. Something to double-check?

Re: VPN performance on a 3800

Chris_Atkinson — Thu, 20 Jul 2023 12:25:17 GMT

That's a bit generic... E.g. No one wants 3DES. Whereas the AES-NI friendly protocols are better on both fronts.

Re: VPN performance on a 3800

the_rock — Thu, 20 Jul 2023 12:28:02 GMT

Hey Chris,

I had more than 1 instance where TAC suggested that sk on the phone to customers and every single time they got an argument back about security, which is 100% valid. TAC response was always that while its true, it should help the speed. In my view, sort of hard sell for security company....

Re: VPN performance on a 3800

dj0Nz — Thu, 20 Jul 2023 12:29:37 GMT

Yes we checked that, too. It's already "tunnel per gateway pair". Only a hadful of VPNs and about 30 firewall rules...

Re: VPN performance on a 3800

the_rock — Thu, 20 Jul 2023 12:31:49 GMT

That setting is usually checked for permanent tunnel...is that the case? Or is it just regular tunnel?

Andy

Re: VPN performance on a 3800

Chris_Atkinson — Thu, 20 Jul 2023 12:34:24 GMT

Understood. Specific issues aside VPN performance may require changes for best results is the point and can also be dependent on the testing methodology.

Hope the SR is resolved for you quickly.

Re: VPN performance on a 3800

Tal_Paz-Fridman — Thu, 20 Jul 2023 12:35:58 GMT

Do you know how many SNDs are there were on the machine, and what is the CPU usage on those SNDs when the issue happened?

Re: VPN performance on a 3800

Chris_Atkinson — Thu, 20 Jul 2023 12:40:18 GMT

We all have different objectives & situations/ constraints that we're dealing with. Security is a sound argument but often doesn't fly if an appliance is undersized for the task at hand.

Ultimately it's a balance like most things.

Re: VPN performance on a 3800

the_rock — Thu, 20 Jul 2023 12:49:21 GMT

I get it, but if you were a customer and security vendor told you that, Im sure you would not be too happy about it.

Re: VPN performance on a 3800

dj0Nz — Thu, 20 Jul 2023 12:52:27 GMT

This is a permanent tunnel connecting to Zscaler. We configured it according to sk174848 and it has been working fine for months. Customer did not change anything but all of a sudden packet loss started last monday.

My guess is that traffic never got above ~350 Mbps but no one noticed. Today, after changing P2 integrity from sha256 to sha1 we noticed rates above 500 Mbps with cpu 0 load at about 50% which is okay.

Currently we're waiting for R&D to come back with suggestions on how to switch back to sha256. 😉

Re: VPN performance on a 3800

the_rock — Thu, 20 Jul 2023 12:57:26 GMT

K, got it, fair enough : - )

Here is what escalation guy from Dallas gave me few months ago when customer had similar issue...client never implemented it, since they had more pressing projects on the go, but not sure if this is something that would help though

Andy

https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk101219&partition=Basic&product=IPSec

fw_clamp_tcp_mss_control - change in Guidbedit to true

mss_value - change to 1200

fw_clamp_vpn_mss -> # fw ctl set int fw_clamp_vpn_mss 1

sim_clamp_vpn_mss -> change to 1 $PPKDIR/conf/simkern.conf

Re: VPN performance on a 3800

dj0Nz — Thu, 20 Jul 2023 13:00:30 GMT

Absolutely right! I wouldn't do that either but, you might know how "urgent" things can get. 8)