R80.10 gateway, can't set sim_clamp_vpn_mss

[Johan_Hillstrom]

Hi

We recently went from R75.46 to R80.10 on a new cluster.

But now we are experiencing IPSec VPN issues, mostly with Azure VPN gw.

We have verified that this is an MTU/MSS issue by temporarily lowering MTU on one of our AD DCs in-house as well as one of the Azure AD servers. However that is not a desirable configuration in the long run.

After plowing through a bunch of SKs I have concluded that what we need to do is enable the sim_clamp_vpn_mss kernel parameter.

Following instructions in this SK doesn't work, even if it says that it applies to R80.10

New VPN features in R77.20 

So how can we enable sim_clamp_vpn_mss?

Is it as simple as using GuiDBedit?

Here are the relevant settings from one of the cluster gateways:

Edited simkern.conf and rebooted, no effect.

# @cat $PPKDIR/boot/modules/simkern.conf
sim_clamp_vpn_mss=1

# fw ctl get int fw_clamp_vpn_mss
fw_clamp_vpn_mss = 1

# fw ctl get int sim_clamp_vpn_mss
fw: Get operation failed: failed to get parameter

# fw ctl get int fw_clamp_tcp_mss
fw_clamp_tcp_mss = 0

# fw ctl get int fw_clamp_tcp_mss_control
fw: Get operation failed: failed to get parameter

# fw ctl get int mss_value
fw: Get operation failed: failed to get parameter

# fw ctl get int sim_ipsec_dont_fragment
fw: Get operation failed: failed to get parameter

# fw ctl get int sim_keep_DF_flag
fw: Get operation failed: failed to get parameter

Any ideas?

Hakan Palmryd

[Vladimir]

Hakan,

I recall changing these settings when setting up the tunnels to AWS VPG.

According to the document on AWS describing VPN with Check Point:

To enable TCP MSS clamping

1. Navigate to the following directory: C:\Program Files (x86)\CheckPoint\SmartConsole\R77.10\PROGRAM\.

2. Open the Check Point Database Tool by running the GuiDBEdit.exe file.

3. Choose Table, Global Properties, properties.

4. For fw_clamp_tcp_mss, choose Edit. Change the value to true and choose OK.

The source could be found here:

Example: Check Point Device without Border Gateway Protocol - Amazon Virtual Private Cloud 

[Johan_Hillstrom]

Hi Vladimir.

Thanks for your quick reply.

The problem is that we only want to enable MSS clamping on VPN tunnels, not globally on the gateway.

Also, the "sim_clamp_vpn_mss"  parameter is not defined at all, searching for it in GuiDBEdit yields nothing.

(However fw_clamp_vpn_mss is enabled, so disabling SecureXL would probably work, but obviously that is not a desired solution)

So it is still a mystery how to define and enable this setting in R80.10...

/Johan

[Vladimir]

Johan,

I am as well uncertain if it is possible to implement it exclusively on tunnels, but considering that creation of VTIs will disable CoreXL, maybe using Azure's (or AWS), default VPN gateways is not the best option unless you have dedicated on-premises check point gateways allocated for cloud connectivity specifically.

[Timothy_Hall]

sim_clamp_vpn_mss definitely exists in the simmod SecureXL driver in R80.10, so you have the right variable name and it hasn't changed.  Unfortunately there is no way to query the live value of SecureXL variables, unlike doing fw ctl get int for the INSPECT driver.  Looks like you have verified the proper variables are set for clamping in the INSPECT driver.

I would suggest running "sim vpn off; fwaccel off; fwaccel on" and see if the situation improves.  That command will force all VPN traffic into INSPECT (where it will hopefully get clamped correctly) and keep SecureXL from handling it at all.

--
My book "Max Power: Check Point Firewall Performance Optimization"
now available via http://maxpowerfirewalls.com.

[Johan_Hillstrom]

Thanks for your insight on this.

Yeah, crippling SecureXL would obviously verify that we are correct in our assumptions.

Not sure what the performance hit would be?

We run about a dozen IPSec tunnels across the globe...

Anyhow, I will disable VPN acceleration now and see if thing look better.

/Johan

[Timothy_Hall]

Those commands don't really cripple SecureXL, it will still do all acceleration except site to site VPN.  Shouldn't be a huge performance hit.  Also fixed syntax in above commands.

--
My book "Max Power: Check Point Firewall Performance Optimization"
now available via http://maxpowerfirewalls.com

[Johan_Hillstrom]

Just nitpicking, but syntax appears to have changed to fwaccel on|off

Change verified by fwaccel stat, showing "Cryptography Features Mask : not available"

Let's see if it works.

/Johan

[Johan_Hillstrom]

SUCCESS!

Following Tim Hall‌ 's suggestions appear do have done the trick.

Running 'tcpdump -i <IF-name> host <a.b.c.d> and "tcp[13] == 2"' now shows MSS adjusted down to 1366 as opposed to 1460

Lets just hop the AD-replications complete successfully now...

/Johan

[Timothy_Hall]

Glad to hear it, however it sounds like MSS clamping (or setting the variable) has a problem with SecureXL for R80.10.  Are you running the latest R80.10 GA jumbo hotfix?  There don't seem to be any fixes related to your problem in the jumbo HFAs but I thought I'd ask anyway.

--
My book "Max Power: Check Point Firewall Performance Optimization"
now available via http://maxpowerfirewalls.com

[Johan_Hillstrom]

Yes, up-to-date on the maintrain release, no custom HFs.

# fw ver
This is Check Point's software version R80.10 - Build 423
# fwaccel ver
Firewall version: R80.10 - Build 025
Acceleration Device: Performance Pack
Accelerator Version 2.1
Firewall API version: 2.91NG (15/5/2014)
Accelerator API version: 2.91NG (15/5/2014)
#

[Timothy_Hall]

OK hopefully Dameon Welch Abernathy will see this thread and can alert the SecureXL team about this possible issue.

--
My book "Max Power: Check Point Firewall Performance Optimization"
now available via http://maxpowerfirewalls.com

[PhoneBoy]

I eventually get to all the threads on CheckMates

While I will alert the relevant R&D folks, your best bet is to open a TAC case: Contact Support | Check Point Software 

[Johan_Hillstrom]

I think I'll drop this in the Swedish SEs' lap... 😉

Ping https://community.checkpoint.com/people/hakane93c2d47-872d-4ed8-a523-121a5b601b8e‌ https://community.checkpoint.com/people/fredre96b5099-96e3-3899-b4f8-9f1cb401955d‌

[Timothy_Hall]

One last thing: the sim vpn off command will not survive a reboot.  To make it persistent add sim_is_vpn_disabled=1 to $PPKDIR/boot/modules/simkern.conf.  Because there is no way to pull the live state of SecureXL variables, I'm curious to see if adding this variable works properly in your environment, such that after a reboot fwaccel stat says that cryptography features are disabled which can easily be checked. 

I find it odd that the clamping function in SecureXL is suddenly broken in R80.10, and wonder if the sim_clamp_vpn_mss variable is just not being set properly at boot time.  Might be interesting to run an od -c $PPKDIR/boot/modules/simkern.conf to make sure there are no unprintable characters messing up the parsing of the file at boot, also did you see any sim errors written to syslog at boot time?  Pretty sure that's where simmod would complain if it had a boot-time problem parsing the simkern.conf file.

--
My book "Max Power: Check Point Firewall Performance Optimization"
now available via http://maxpowerfirewalls.com

[Johan_Hillstrom]

Hi

Unfortunately, sim vpn off led to a lot of problems with our tunnels to older GWs, both R77.30 and R75.46 (I know, we SHOULD update)

Really weird behavior where we saw outgoing traffic getting encrypted, but 0 packets coming back.

And at the far end of the tunnel we saw exactly nothing.

So I guess we completely broke MSS, leading to silent drops..

I have started the process of creating a TAC, will post case # here when we get it. (On Collaborative support ;-(

Here is the od -c, found no sim related errors, will do a reboot of passive gateway later tonight.

# od -c $PPKDIR/boot/modules/simkern.conf

0000000   s   i   m   _   c   l   a   m   p   _   v   p   n   _   m   s

0000020   s   =   1  \n

0000024

/Johan

[CP-NDA]

Hi,

We had a similar issue and here is how we solved it with TAC:

1. Enable MSS Clamping on the Gateway for VPN
fw ctl set int fw_clamp_vpn_mss 1

If SecureXL is enable you have to enable MSS Clamping in SecureXL
vi $PPKDIR/boot/modules/simkern.conf
sim_clamp_vpn_mss=1

Reboot the GW . I don't know how to reload this parameter on fly

2. Decrease MSS value via GUIDbEdit
Set "mss_value" to the desired value (1360 in our case) on each involved interface in the VPN communication

Nicolas

[Worapong_Janloy]

To adjust mss_value in cluster env I have to adjust on all gw object or cluster object? 

[Johan_Hillstrom]

Please see previous comments, you need to adjust settings on all gateway objects.

[Johan_Hillstrom]

Sorry for late feedback on this.

We got the same answer as Nicolas from TAC.

Enabling fw_clamp_vpn_mss, AND manually specifying mss_value (we set 1350 per MS recommendations)

Previously we had mss_value set to 0 to facilitate automatic calculation of MSS.

Strange thing is that this was working properly on R75.46, so apparently something has changed when it comes to automatically calculating MSS.

[Ilmo_Anttonen]

Johan, thanks for the update.

Did you set mss_value globally or on specific interfaces? I'm curious to learn if there are any negative side effects from setting it globally.. I have a client with the same issue as described in OP, except that we were able to set all values but with no effect.

[CP-NDA]

Yes we set it globally on each interfaces involved in the VPN communication (External / Internal / DMZ / ...)

[Ilmo_Anttonen]

Nicolas, thanks for the answer. I'm not sure I understand though. The only option I found in GuiDBedit under Global Properties > Properties is the "mss_value". Setting that value changes all interfaces globally, right? Or am I misunderstanding? I only want to change MSS for the interfaces involved in the VPN commmunication. Where do you set that?

[Johan_Hillstrom]

Hi Ilmo

Sorry for late reply, still on holidays 😉

I will try to clarify.

First, enable MSS Clamping on the Gateway for VPN, to activate MSS clamping on VPN only, not affecting normal traffic.
fw ctl set int fw_clamp_vpn_mss 1

Then you need to set mss_value to the specific value needed (in our case 1350)

This has to be done on all involved network interfaces on all gateway objects. In our case we modified it on LAN and External.

You should find the mss_value already defined on each interface in GuiDBedit, but set to 0 (meaning it will auto-adjust)

Hope this helps.

[Ilmo_Anttonen]

Yes OK I got it! Thanks!

I just had a brief look in the GuiDBedit tool when I was at the customer premises and at the time only found the mss_value under global properties. I now understand there is such a label under each interface. Haven't got much experience with the GuiDBedit tool and no lab available.. hence the stupid questions.

Once again thank you Johan and Nicolas.

[Worapong_Janloy]

How can I verify mss value on my gateway after I changed mss_value by GuiDBedit tool.

[Johan_Hillstrom]

You can verify it using a simple ping test using different payload sizes. 

MTU/MSS - How to test using PING - Tech-Wiki 

[PhoneBoy]

After a policy install, I think you should also see the value reflected in the output of the command fw ctl get fw_clamp_tcp_mss

[Ari_Rentola]

At R80.20:

/opt/CPppak-R80.20/boot/modules/simkern.conf has text:

# Deprecated location.
# Any change should be made at /opt/CPppak-R80.20/conf/simkern.conf