Policy push causes vpn tunnel to go down in R80.30

[the_rock]

Hello everyone,

I was hoping someone might be able to give some advice/suggestion about this problem. I had been working with a customer who is running 2 sets of firewalls (one R80.30 single firewall) and other R80.30 cluster (a-p) and single box has site to site vpn with Cisco asa and works fine, no issues at all, but the cluster, site to site vpn tunnel with a different cisco asa, whenever they push policy to it, tunnel goes down and comes back on its own maybe 30 mins later. Weird thing is, we had them select "keep ike sas" option in global properties and also check "keep all connections" from connection persistence, but no luck.

I did not wish to have them change any supernet stuff in guidbedit, since Cisco never said they saw anything coming to their end with wrong network. Anyone seen this problem in R80.xx versions at all? When we did ike debug and zdebug, there were no drops and ike.elg when reviewed in ikeview, did not even show this specific tunnel (this was all when problem was happening). We are waiting for Cisco side to confirm whet exactly they see on their end when problem happens, but in the meantime, maybe someone can suggest something for us to try on CP side 🙂

Thanks a lot in advance!

By the way, we have tac case open now for 23 days and no good advice really came from it. Not sure if its still with 2nd level or with escalations.

Thanks!

Andy

[PhoneBoy]

This feels like something on the Cisco end based on the description you provided, but hard to prove.
The fact you didn’t even see the relevant tunnel in IKE debug is definitely odd.

[the_rock]

 I dont believe its Cisco side. Here are drops on CP end when issue is happening:

# fw ctl zdebug + drop | grep 205.207.130.253

@;127721520;[cpu_0];[SIM-204987365];sim (vpn_encrypt): drop due vpn_ipsec_encrypt returns PKT_DROP(3), conn: <172.16.193.243,50846,205.207.130.253,22,6>;
@;127721520;[cpu_0];[SIM-204987365];handle_vpn_encryption: ipsec_encrypt failed: failed to find SA. Dropping packet... conn: <172.16.193.243,50846,205.207.130.253,22,6>;
@;127721520;[cpu_0];[SIM-204987365];sim_pkt_send_drop_notification: (0,1) received drop, reason: Encryption Failed, conn: <172.16.193.243,50846,205.207.130.253,22,6>;
@;127721520;[cpu_0];[SIM-204987365];sim_pkt_send_drop_notification: sending packet dropped notification drop mode: 0 debug mode: 1 send as is: 0 track_lvl: -1, conn: <172.16.193.243,50846,205.207.130.253,22,6>;
@;127721520;[cpu_0];[SIM-204987365];sim_pkt_send_drop_notification: sending single drop notification, conn: <172.16.193.243,50846,205.207.130.253,22,6>;
@;127721521;[cpu_0];[SIM-204987365];do_packet_finish: SIMPKT_IN_DROP vsid=0, conn:<172.16.193.243,50846,205.207.130.253,22,6>;
@;127722353;[cpu_0];[SIM-204987365];sim (vpn_encrypt): drop due vpn_ipsec_encrypt returns PKT_DROP(3), conn: <172.16.193.243,50846,205.207.130.253,22,6>;
@;127722353;[cpu_0];[SIM-204987365];handle_vpn_encryption: ipsec_encrypt failed: failed to find SA. Dropping packet... conn: <172.16.193.243,50846,205.207.130.253,22,6>;
@;127722353;[cpu_

[the_rock]

Forgot to mention, we also ended up completely disabling supernet globally AND also per that specific community, same problem.

[stuart2020]

Hello,

We are experiencing a very similar issue with a VPN to an AWS gateway. Our Checkpoint is running R80.40 take 87. Did you find a fix for this issue?

Thank you. 

[the_rock]

As a matter of fact, we did : ). We ended up turning of vpn acceleration for that tunnel...vpn accel off x.x.x.x y.y.y.y (x.x.x.x would be cp external vpn IP and y.y.y.y would be remote peer). Once that was done and policy pushed, it worked. Now, here is the thing...in our case it was fine, since not a lot of traffic through that tunnel, but if thats not the case with your tunnel, maybe disabling vpn acceleration might not be a god idea...just saying. But, either way, worth a try.

Andy

[stuart2020]

Excellent, I will look at giving that a try. 

[the_rock]

Sounds good! Personally, I found acceleration to be problem on CP for the last 15 years and there has never been a solid, logical solution to it, sadly. Anyway, give it a go and see what happens.

[stuart2020]

I gave that a try but unfortunately it didn't resolve my issue. The tunnel still dropped for a few minutes after the policy had been pushed. Back to the drawing board. I will get a ticket raised with Checkpoint support and see what they suggest.  

[the_rock]

If that failed, then you can try fwaccel off as well in conjunction with vpn accel off and see what happens. If that failed as well, then yes, raise a case with TAC.

Andy

[stuart2020]

Good idea, I will give that a try as well.

[the_rock]

If that fails too, Im really sorry brother, Im not a good liar, I got nothing else : (. What I would personally do in that case is go with vpn debug (ike.elg)

vpn debug ikeon

run quick test when issue is there, then turn off the debug (vpn debug ikeoff) and look for ike.elg file in $FWDIR/log directory.

Im happy to do remote over the weekend if you like, not like I do anything else during covid these days anyway, haha. Let me know...you are welcome to message me privately as well.

Andy

[Chris_Atkinson]

Please also review sk142355 in case it is relevant to your scenario.

[the_rock]

Im pretty sure he did that already, but let Stuart confirm.

[stuart2020]

Thanks Andy, That was the first SK I tried but it didn't fix it. I collected some ike, vpnd and tcpdump debugs yesterday but I can't see anything obvious. I will upload them to Checkpoint to see if they can spot anything. 

[vladdar]

vpn accel off x.x.x.x helped me also with the same issue altough I dind't see any drops with fw ctl zdebug + drop

i am on R81.10.15