topic Re: VPN disturbances in General Topics

VPN disturbances

Moudar — Thu, 12 Sep 2024 09:05:41 GMT

Hi,

I’ve received reports from clients experiencing unstable VPN connections (remote clients).

Upon reviewing the VPND logs, I found the following:

Unable to open '/dev/fw6v0': No such file or directory Unable to open '/dev/fw6v0': No such file or directory Unable to open '/dev/fw6v0': No such file or directory SvcSk_close: refraining from closing socket -1 httpMime_destroy_db: Destroying the MIME database. [vpnd 2393]@fw01[10 Sep 11:05:18] Warning:cp_timed_blocker_handler: A handler [0x80e18e0] blocked for 6 seconds. [vpnd 2393]@fw01[10 Sep 11:05:18] Warning:cp_timed_blocker_handler: Handler info: Library [vpnd], Function offset [0x1e8e0]. [vpnd 2393]@fw01[10 Sep 11:05:18] Warning:cp_timed_blocker_handler: A handler [0xf6019d40] blocked for 6 seconds. [vpnd 2393]@fw01[10 Sep 11:05:18] Warning:cp_timed_blocker_handler: Handler info: Library [/opt/CPshrd-R81.20/lib/libmessaging.so], Function offset [0x1d40]. [vpnd 2393]@fw01[10 Sep 11:05:18] Warning:cp_timed_blocker_handler: A handler [0xf5a82d50] blocked for 6 seconds. [vpnd 2393]@fw01[10 Sep 11:05:18] Warning:cp_timed_blocker_handler: Handler info: Library [/opt/CPshrd-R81.20/lib/libComUtils.so], Function offset [0x1f350]. Unable to open '/dev/fw6v0': No such file or directory Unable to open '/dev/fw6v0': No such file or directory Unable to open '/dev/fw6v0': No such file or directory

What factors can cause a handler to become blocked?

What is the /dev/fw6v0 device, and why might it be missing?

Could a blocked handler impact the VPN client experience, potentially causing connection issues?

Re: VPN disturbances

G_W_Albrecht — Thu, 12 Sep 2024 09:19:37 GMT

What is the issue in detail, unstable RA VPN usually means disconnect and reconnect ? I found sk181526: Intermittent authentication failures to AzureAD on an Identity Awareness Seurity Gateway with this error message.

Re: VPN disturbances

Moudar — Thu, 12 Sep 2024 09:30:32 GMT

The link is about Identity Awareness (pdpd.elg)! Even though you have right it is the same log!

Yes, disconnect and reconnect

Re: VPN disturbances

PhoneBoy — Thu, 12 Sep 2024 21:09:01 GMT

What version/JHF level?
Perhaps a Super Seven output would be useful here, as it could be performance related: https://community.checkpoint.com/t5/Scripts/S7PAC-Super-Seven-Performance-Assessment-Commands/m-p/40528#M703

Re: VPN disturbances

Moudar — Thu, 12 Sep 2024 21:15:10 GMT

81.20 take 76

Re: VPN disturbances

Moudar — Fri, 13 Sep 2024 07:12:38 GMT

I've now run the 7-command script. What stands out to me as particularly interesting is:

Accelerated conns/Total conns : 105/52071 (0%)

here is fawaccel stat:

Rule 62 is for SMTP, and it's challenging to move it down!

here is fwaccel stats -s:

fwaccel stat -s fwaccel: illegal option -- s Invalid option '?' [Expert@fw01:0]# fwaccel stats -s Accelerated conns/Total conns : 105/49378 (0%) LightSpeed conns/Total conns : 0/49378 (0%) Accelerated pkts/Total pkts : 46429315584/53547009472 (86%) LightSpeed pkts/Total pkts : 0/53547009472 (0%) F2Fed pkts/Total pkts : 7117693888/53547009472 (13%) F2V pkts/Total pkts : 255709233/53547009472 (0%) CPASXL pkts/Total pkts : 2084716164/53547009472 (3%) PSLXL pkts/Total pkts : 42403141530/53547009472 (79%) CPAS pipeline pkts/Total pkts : 0/53547009472 (0%) PSL pipeline pkts/Total pkts : 0/53547009472 (0%) QOS inbound pkts/Total pkts : 0/53547009472 (0%) QOS outbound pkts/Total pkts : 0/53547009472 (0%) Corrected pkts/Total pkts : 0/53547009472 (0%)

so, any suggestions?

Re: VPN disturbances

PhoneBoy — Fri, 13 Sep 2024 12:56:57 GMT

What exactly is the rule that is disabling templates?
A screenshot (with sensitive details blurred) will help.

The fact that (almost) no connections are accelerated is at least contributing to the issue, if not the actual cause.

Re: VPN disturbances

Moudar — Fri, 13 Sep 2024 13:12:29 GMT

It is a rule that allows connection (source) from Microsoft Outlook worldwide among other services to our Mail servers (destination) with services and application = smtp

Re: VPN disturbances

the_rock — Fri, 13 Sep 2024 13:30:51 GMT

You can try disable securexl and test. If that works, then I would open TAC case and investigate further.

Andy

Re: VPN disturbances

PhoneBoy — Fri, 13 Sep 2024 14:21:42 GMT

Even though it's not documented anywhere, I suspect it's the object in the Destination field (Logical Server) that is causing traffic to not be templated.

I have an idea of how to work around this, but it involves creating an inline layer (change action from Accept to an Inline Layer, create a new one with just Firewall blade active).
The top level rule involves a regular host object (ls_smtp_ip) that is the same IP as ls_smtp (the Logical Server object).
The UI will give you a warning when you create an object with the same IP (which is expected).
The inline layer will only be evaluated if the top level rule matches, which is why the rules in the inline layer aren't specific.
When it's all said and done, it should look something like this:

What should happen (assuming installing policy isn't blocked with this configuration) is that the traffic matched by rules below this should now template.
That would mean the percentage of Accelerated Connections should dramatically increase.

Re: VPN disturbances

PhoneBoy — Fri, 13 Sep 2024 14:24:19 GMT

Doubt it will help in this case since almost none of his connections are accelerated as it is and disabling SecureXL only disables the templating.

Re: VPN disturbances

the_rock — Fri, 13 Sep 2024 14:25:18 GMT

Makes sense.

Re: VPN disturbances

Moudar — Sun, 15 Sep 2024 21:43:15 GMT

Only Firewall is chosen in the inline layer settings!

Attempting to move the rule down fails due to the presence of multiple inline layers:

So, what other options do we have?

Re: VPN disturbances

Tal_Paz-Fridman — Sat, 14 Sep 2024 10:52:42 GMT

Are you able to install policy with this Validation Error? It should not allow to Publish or Install Policy

Re: VPN disturbances

the_rock — Sat, 14 Sep 2024 11:54:34 GMT

I dont think it has anything to do the with the blade selected, its telling you that object can NOT be used in that inline layer, for whatever reason. Though, if you check solution from the sk indicated, its pretty clear why.

Andy

Re: VPN disturbances

Moudar — Sat, 14 Sep 2024 12:19:14 GMT

No, the policy failed to install with the above error message

Re: VPN disturbances

PhoneBoy — Mon, 16 Sep 2024 12:40:42 GMT

I remembered that SK existed, but forgot to check it for suggesting this workaround.

Sadly, the only other suggestion I have is to not use a Logical Server object.
With almost all of your connections not templating, you'll probably have other performance-related issues down the road.

Re: VPN disturbances

Moudar — Mon, 16 Sep 2024 13:03:42 GMT

That's unfortunate. How do other companies address this issue?

Re: VPN disturbances

PhoneBoy — Mon, 16 Sep 2024 13:24:42 GMT

Logical Server objects are rarely used, therefore the limitations they impose are rarely encountered.
Load balancing access to SMTP servers is usually done via DNS.

Re: VPN disturbances

Moudar — Fri, 20 Sep 2024 08:38:40 GMT

We have now removed the logical server object, and the SecureXL status currently appears as follows:

It has been running for 3 days, but the "Accelerated conns/Total conns" ratio remains very low:

fwaccel stats -s Accelerated conns/Total conns : 1013/58378 (1%) LightSpeed conns/Total conns : 0/58378 (0%) Accelerated pkts/Total pkts : 66915308684/75883929440 (88%) LightSpeed pkts/Total pkts : 0/75883929440 (0%) F2Fed pkts/Total pkts : 8968620756/75883929440 (11%) F2V pkts/Total pkts : 357709365/75883929440 (0%) CPASXL pkts/Total pkts : 2824937581/75883929440 (3%) PSLXL pkts/Total pkts : 61384916652/75883929440 (80%) CPAS pipeline pkts/Total pkts : 0/75883929440 (0%) PSL pipeline pkts/Total pkts : 0/75883929440 (0%) QOS inbound pkts/Total pkts : 0/75883929440 (0%) QOS outbound pkts/Total pkts : 0/75883929440 (0%) Corrected pkts/Total pkts : 0/75883929440 (0%)

The gateway is 6500:

This is Check Point CPinfo Build 914000248 for GAIA [MGMT] HOTFIX_R81_20_JUMBO_HF_MAIN Take: 65 [IDA] No hotfixes.. [CPFC] No hotfixes.. [FW1] HOTFIX_R81_20_JHF_T53_BLOCK_PORTAL_MAIN Take: 2 HOTFIX_GOT_TPCONF_AUTOUPDATE HOTFIX_R80_40_MAAS_TUNNEL_AUTOUPDATE HOTFIX_R81_20_JUMBO_HF_MAIN Take: 65 HOTFIX_PUBLIC_CLOUD_CA_BUNDLE_AUTOUPDATE

any ideas?!