Not able to establish BGP when ISP failover, BGP is using Configured VTI

[ajsingh]

Hi,

I have a CP to CP site to site VPN tunnel which has one VTI configured (numbered) and over that BGP is established. 

One CP has 2 isp configured and ISP redundancy has been configured and apply to VPN settings too. 

Upon failover of ISP , Internet traffic is working. VPN tunnel seems to be up too with Peer (CP) but BGP is in Active state. I couldn't ping other side VTI ip also. 

I can see traffic is going from both sides to each other via VTI and encrypted but none of the sides are replying. IS there any limitation or am i missing anything in the configuration to make this setup work ?

Both FW's are R81.20 + T105 . . Managed by same FW manager. 

[the_rock]

Do you have simple diagram? It would help, for sure.

Andy

[ajsingh]

[ajsingh]

I do see Peer CP FW recieved ping request and reply :

[vs_0][fw_0] eth0:i[44]: 10.2.2.2 -> 10.1.1.1 (ICMP) len=84 id=38959

ICMP: type=8 code=0 echo request id=35083 seq=15

[vs_0][fw_0] vpnt1:I[44]: 10.2.2.2 -> 10.1.1.1 (ICMP) len=84 id=38959

ICMP: type=8 code=0 echo request id=35083 seq=15

[vs_0][fw_0] vpnt1:o[44]: 10.1.1.1 -> 10.2.2.2 (ICMP) len=84 id=42176

ICMP: type=0 code=0 echo reply id=35083 seq=15

[vs_0][fw_0] vpnt1:O[44]: 10.1.1.1 -> 10.2.2.2 (ICMP) len=84 id=42176

ICMP: type=0 code=0 echo reply id=35083 seq=15

[vs_0][fw_1] vpnt1:Oe[44]: 10.1.1.1 -> 10.2.2.2 (ICMP) len=84 id=42176

ICMP: type=0 code=0 echo reply id=35083 seq=15

But I dont see any reply on on-prem FW VTI when ISP failovers.

[the_rock]

Can you do zdebug for the relevant IP when it fails? fw ctl zdebug + drop | grep x.x.x.x (just replace x.x.x.x with right IP)

Andy

[ajsingh]

Hi,

This is what i see : 

[Expert@GW5400-LAB:0]# fw ctl zdebug + drop | grep 10.1.1.1
@;52686503.19611;[kern];[tid_0];[SIM4];sim (vpn_encrypt): drop due vpn_ipsec_encrypt returns PKT_ERROR(0), conn: <10.2.2.2,22141,10.1.1.1,0,1>;
@;52686503.19612;[kern];[tid_0];[SIM4];handle_vpn_encryption: ipsec_encrypt failed: encrypted packet too big. Dropping packet... conn: <10.2.2.2,22141,10.1.1.1,0,1>;
@;52686503.19613;[kern];[tid_0];[SIM4];sim_pkt_send_drop_notification: (0,1) received drop, reason: Encryption Failed (5), conn: <10.2.2.2,22141,10.1.1.1,0,1>;
@;52686503.19614;[kern];[tid_0];[SIM4];sim_pkt_send_drop_notification: sending packet dropped notification drop mode: 0 debug mode: 1 send as is: 0 track_lvl: -1, conn: <10.2.2.2,22141,10.1.1.1,0,1>;
@;52686503.19615;[kern];[tid_0];[SIM4];sim_pkt_send_drop_notification: sending single drop notification, conn: <10.2.2.2,22141,10.1.1.1,0,1>;
@;52686509.19616;[kern];[tid_0];[SIM4];do_packet_finish: SIMPKT_IN_DROP vsid=0, conn:<10.2.2.2,22141,10.1.1.1,0,1>;
@;52686512.19619;[kern];[tid_0];[SIM4];sim (vpn_encrypt): drop due vpn_ipsec_encrypt returns PKT_ERROR(0), conn: <10.2.2.2,22141,10.1.1.1,0,1>;
@;52686512.19620;[kern];[tid_0];[SIM4];handle_vpn_encryption: ipsec_encrypt failed: encrypted packet too big. Dropping packet... conn: <10.2.2.2,22141,10.1.1.1,0,1>;
@;52686512.19621;[kern];[tid_0];[SIM4];sim_pkt_send_drop_notification: (0,1) received drop, reason: Encryption Failed (5), conn: <10.2.2.2,22141,10.1.1.1,0,1>;
@;52686512.19622;[kern];[tid_0];[SIM4];sim_pkt_send_drop_notification: sending packet dropped notification drop mode: 0 debug mode: 1 send as is: 0 track_lvl: -1, conn: <10.2.2.2,22141,10.1.1.1,0,1>;
@;52686512.19623;[kern];[tid_0];[SIM4];sim_pkt_send_drop_notification: sending single drop notification, conn: <10.2.2.2,22141,10.1.1.1,0,1>;
@;52686514.19624;[kern];[tid_0];[SIM4];do_packet_finish: SIMPKT_IN_DROP vsid=0, conn:<10.2.2.2,22141,10.1.1.1,0,1>;
@;52686536.19627;[kern];[tid_0];[SIM4];sim (vpn_encrypt): drop due vpn_ipsec_encrypt returns PKT_ERROR(0), conn: <10.2.2.2,22141,10.1.1.1,0,1>;
@;52686536.19628;[kern];[tid_0];[SIM4];handle_vpn_encryption: ipsec_encrypt failed: encrypted packet too big. Dropping packet... conn: <10.2.2.2,22141,10.1.1.1,0,1>;
@;52686536.19629;[kern];[tid_0];[SIM4];sim_pkt_send_drop_notification: (0,1) received drop, reason: Encryption Failed (5), conn: <10.2.2.2,22141,10.1.1.1,0,1>;
@;52686536.19630;[kern];[tid_0];[SIM4];sim_pkt_send_drop_notification: sending packet dropped notification drop mode: 0 debug mode: 1 send as is: 0 track_lvl: -1, conn: <10.2.2.2,22141,10.1.1.1,0,1>;
@;52686536.19631;[kern];[tid_0];[SIM4];sim_pkt_send_drop_notification: sending single drop notification, conn: <10.2.2.2,22141,10.1.1.1,0,1>;
@;52686538.19632;[kern];[tid_0];[SIM4];do_packet_finish: SIMPKT_IN_DROP vsid=0, conn:<10.2.2.2,22141,10.1.1.1,0,1>;

[the_rock]

Please check if this sk might apply.

Andy

https://support.checkpoint.com/results/sk/sk166417

[ajsingh]

Thank you. I checked and no it doesnt. I am using IKEv1 and vpn tu tlist does not show any symptoms as the SK :

[Expert@GW5400-LAB:0]# vpn tu tlist

+-----------------------------------------+----------------------------------+---------------------+
| Peer: X.X.X.X - abc | MSA: 7f08383dfp88 | i: 0 ref: 5 |
| Methods: ESP Tunnel PFS AES-256 SHA256..| | i: 1 ref: 2 |
| My TS: 0.0.0.0/0 | | |
| Peer TS: 0.0.0.0/0 | | |
| MSPI: 3e (i: 0, p: 0, d: 0) | Out SPI: b3c35590 | |
| Tunnel created: Oct 7 14:05:57 | NAT-T | |
| Tunnel expiration: Oct 7 15:05:57 | Connected | |
+-----------------------------------------+----------------------------------+---------------------+

[the_rock]

To me, that 100% looks like phase 2 issue, based on the error. Can you send what I attached looks like for your side?

Andy

[ajsingh]

[the_rock]

Try unchecking permanent tunnel option and install policy. if no joy, just put it back and open TAC case, might need some more debugging. Btw, just curious, why use ikev1? Thats insecure, plus, ikev2 is standard nowdays.

Andy

[ajsingh]

Ok , Yes i will do that. it was just for testing but I will do that change as well.

Thanksalot for your time. 

[the_rock]

No need to thank me mate, we are here to help. If you need reference for what I meant, check out below.

Andy

Btw, we can do remote if you allow that.

https://learningnetwork.cisco.com/s/question/0D56e0000C4N9csCQC/ikev1-versus-ikev2

https://docs.fortinet.com/document/fortigate/7.6.0/ssl-vpn-to-ipsec-vpn-migration/883534/ikev1-or-ik...

https://www.sonicwall.com/support/knowledge-base/advantages-of-site-to-site-vpn-with-ikev2-over-ikev...