cancel
Showing results for 
Search instead for 
Did you mean: 
Create a Post
Highlighted
TheRealDiZ
Nickel

SecureXL R80.20 - Issue on ALL High TCP Ports

Hey guys,

 

After upgrade from R77.30 to R80.20, I notice that I got issue on all connections with high TCP ports passing through a VPN tunnel.

That was huuuge... Fortunately after the upgrade I have immediately tried to disable SecureXL acceleration as per https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solut... and solved the issue.

 

Anyone has experienced this issue before?

 

I know that in R80.20 SecureXL was moved to Fw_Worker.

Anyone can explain to me the difference from R77.30 in detail?

I think that probably this mechanism change is causing issue on all connections with high tcp ports.

 

BR

Luca

 

Tags (2)
15 Replies
Admin
Admin

Re: SecureXL R80.20 - Issue on ALL High TCP Ports

First I'll make my standard statement for these situations: if disabling SecureXL solves the problem, open a TAC case.

Not sure what knowing the differences in SecureXL will help in terms of solving the issue at hand, which is most likely a bug.
The changes in SecureXL were made to improve performance and support more than 40 cores.

What rule/service is the relevant traffic matching on?
TheRealDiZ
Nickel

Re: SecureXL R80.20 - Issue on ALL High TCP Ports

Hi @PhoneBoy ,

 

Yessir TAC case already opened.

The traffic is passing through the same VPN tunnel and matching same rule as with the previous installed version R77.30.

No changes were made (SecureXL was always enabled).

 

I'll keep you posted with TAC updates in order to understand if there is a major issue on R80.20.

 

RealD!Z

0 Kudos

Re: SecureXL R80.20 - Issue on ALL High TCP Ports

Note that in R80.20 Jumbo HFA Take 47 and later, a new command called vpn accel was added that allows switching off SecureXL acceleration just for VPN traffic (similar to sim vpn off;fwaccel off;fwaccel on in R80.10 and earlier), which is different from the f2f_addresses mechanism you used:

sk151114: "fwaccel off" does not affect disabling acceleration of VPN tunnels in R80.20 and above

This new command was not available in R80.20 vanilla.  Still as Dameon mentioned it is very important to figure out why SecureXL needs to be disabled in the first place, regardless of the mechanism you use to disable it.  🙂

 

"IPS Immersion Training" Self-paced Video Class
Now Available at http://www.maxpowerfirewalls.com
0 Kudos
Admin
Admin

Re: SecureXL R80.20 - Issue on ALL High TCP Ports

As I don't know what the rule was on R77.30, please share the details of it.

TheRealDiZ
Nickel

Re: SecureXL R80.20 - Issue on ALL High TCP Ports

Hi @PhoneBoy

It seems the problem is related to VPN, because the traffic is passing (NATted) through a VPN S2S.
Before the upgrade there were no issues at all.. I'm investigating it with the TAC.. I'll keep you posted because I think this will be helpful for a lot of people.

By the way this is the fw ctl zdebug error when we got the issue:

"[cpu_0];[SIM-206945184];sim_pkt_send_drop_notification: (0,0) received drop, reason: general reason, conn:"
0 Kudos

Re: SecureXL R80.20 - Issue on ALL High TCP Ports

Hi @TheRealDiZ 

 

I am also experiencing the same issue after upgrade to R80.20. Can you please share if you guys were able to find the reason along with TAC.

TheRealDiZ
Nickel

Re: SecureXL R80.20 - Issue on ALL High TCP Ports

Hi @Raman_Arora ,

 

Unfortunately after one month I still discussing this issue with the TAC... I'll keep you posted!

.. 😞

0 Kudos

Re: SecureXL R80.20 - Issue on ALL High TCP Ports

Hi TheRealDiz,

Did you manage to resolve this issue together with TAC?
0 Kudos
TheRealDiZ
Nickel

Re: SecureXL R80.20 - Issue on ALL High TCP Ports

Hi @geir ..

Nope Debugs on Debugs on Debugs.... SecureXL + R80 are issues on issues on issues.. for real guys issues on different environement but result is the same..Disable the SecureXL.. Sorry but I'm disappointed.
0 Kudos

Re: SecureXL R80.20 - Issue on ALL High TCP Ports

Hi guys,

We're experiencing the same kind issue with VPNs and SecureXL. Turn accel off and it starts working.

The funny thing is ICMP and port 3306 works but any other port we've tried fails. This issue manifests when routing from one VPN to another. We have two VTIs configured and we resolved the issues from on-prem by turning the accel off just for those two nodes:
vpn accel off xxx.xxx.xxx.xxx xxx.xxx.xxx (public IP addresses of the remote VPN tunnels you want to turn off)

But with this method, you can only turn off accel just for two nodes

Hope this helps someone.

Please keep up posted if a solution comes about

0 Kudos

Re: SecureXL R80.20 - Issue on ALL High TCP Ports

Hi Serban,

Thanks for the update, I think you mean vpn accel off xxx.xxx.xxx.xxx xxx.xxx.xxx not fw accel off xxx.xxx.xxx.xxx xxx.xxx.xxx.

 

"IPS Immersion Training" Self-paced Video Class
Now Available at http://www.maxpowerfirewalls.com
0 Kudos

Re: SecureXL R80.20 - Issue on ALL High TCP Ports

Hi Tim,

In R80.20 jumbo hotfix Take 47 they introduced a command to turn off the acceleration just for some VPNs based on remote end IP address. This is limited to just two IP addresses.

Here's a snip of the CP docs:
Overview
A new CLI interface called vpn accel has been introduced for controlling acceleration of VPN tunnels.

The vpn accel interface only affects VPN connections, and does not have any impact on clear-text traffic.

The vpn accel interface supports disabling acceleration of VPN tunnels per VPN peer.



Usage
The behavior of VPN acceleration from R80.20 and above will be controlled by the following options:



syntax: vpn accel [SWITCH_OPTION] [ALLOW_OPTION]



SWITCH_OPTIONS:

off [peer_1_ip] [peer_2_ip] ............. # switch off vpn acceleration (see Notes below)

on ................................................... # switch on vpn acceleration for all peers

stat ................................................ # show vpn acceleration status



ALLOW_OPTION:

-y .................................................... # do not prompt for confirmation



Disabling acceleration of VPN per peer versus globally disabling acceleration of VPN
The command allows you to disable acceleration of VPN tunnels for a given peer by typing the peer IP address (supports disabling acceleration for up to two peers). When acceleration is switched off for given peers, this will cause the deletion of the IPSEC SAs associated with them.
If no address was provided, the acceleration will be switched off for all peers and it will cause the deletion of all IPSEC SAs immediately.


Reenabling acceleration of VPN tunnels
Reenabling acceleration of VPN tunnels after switching off acceleration for given peers will cause the deletion of all IPSEC SAs that belong to these peers.
If no address was provided when acceleration is switched off, reenabling acceleration of VPN tunnels will cause the immediate deletion of all IPSEC SAs.


How to persistently disable VPN acceleration (will survive reboot)
Refer to sk148872 - VPN Traffic drops due to decryption error "Tunnel is accelerated but packet was not decrypted by SecureXL"



Impact on VPN connections
By disabling tunnel acceleration and specifying a particular peer, every VPN accelerated connection with this peer will be lost.
By disabling tunnel acceleration without specifying a particular peer, every VPN accelerated connection with any peer will be lost.


Limitations and Important Notes
In R80.20, 'vpn accel' is supported since R80.20 jumbo hotfix Take 47.
The feature does not support disabling acceleration of VPN tunnels of a specific peer IP if it is a DAIP gateway.
When you run 'vpn accel on' (enabling acceleration of VPN tunnels), fwaccel status will not be effected. This means that new VPN tunnels cannot be accelerated if SecureXL has been stopped. You can check the status by running 'fwaccel stat'.
Each time the command 'vpn accel off' is run, the non-accelerated peers list will be overridden with the last IPs that were inserted.
Packets of accelerated VPN connections will be dropped by SecureXL. The following drops are legitimate at the time of disabling VPN acceleration:
1. " sim (vpn_encrypt): drop due vpn_ipsec_encrypt returns PKT_DROP(3) ----> handle_vpn_encryption: ipsec_encrypt failed. Dropping packet "

2. " forward_before_encrypt: No forwarding destination before encrypt ----> do_packet_finish: SIMPKT_IN_DROP "

3. " invalid in_corr_info => dropping packet, in_corr_info(sxl_dev_id:32, flags:0x0, pkt_flags:0x0) "



Examples
'vpn accel off': All existing Tunnels (IPSEC SAs) will be deleted and become non-accelerated upon renegotiation (new IPsec SA establishment).
'vpn accel off 192.168.15.200 172.23.43.54': Tunnels (IPSEC SAs) of given peers (192.168.15.200,172.23.43.54) will be deleted and become non-accelerated upon renegotiation (new IPsec SA establishment).
'vpn accel on': All existing non-accelerated tunnels (IPSEC SAs) will be deleted and become accelerated upon IPsec SA establishment (if algorithms and other tunnel parameter allow that)

Re: SecureXL R80.20 - Issue on ALL High TCP Ports

Sorry, I read afterward. yes you're right VPN ACCEL OFF

 

*** Corrected so people don't get confused and turn off SecureXL altogether 😄 ***

TheRealDiZ
Nickel

Re: SecureXL R80.20 - Issue on ALL High TCP Ports

Hi @Serban_Biliuti ,

 

That's HUGE and very useful and I will try to use it.. Thank you!

By the way we are still investigating why with the accelaration we got the issue.

 

I'll keep you guys posted with my findings with the TAC.

We actually need to run other debugs.

 

 

0 Kudos

Re: SecureXL R80.20 - Issue on ALL High TCP Ports

*** Update ***
We managed to fix our issue with VPN-to-VPN routing not working on almost all ports.
We had previously turned off accel for one VPN using the "vpn accel off xxx.xxx.xxx.xxx" command to fix the issue. Then we started having issues with traffic from other VPNs that were still accelerated.
So we took a chance and re-enabled all acceleration on VPNs. That seems to have solved the issue.
There was a suggestion from Checkpoint that it looked like an issue with traffic coming from an accelerated VPN to a non-accelerated one.
We did some changes during this time to correct our setup but it still doesn't make sense why the traffic would behave differently based on ports (ICMP and 3306 works, all the other ones fail).

Bottom line is: Traffic works fine now and all VPNs are accelerated.

The traffic still shows up different with fw monitor -F "0,0,0,0,0" when you change ports but at least it works.

Left the TAC opened for CheckPoint to figure it out.