topic Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination in Firewall and Security Management

Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

cyberluke365 — Wed, 30 Oct 2024 09:56:20 GMT

Hello,

I have a site-to-site VPN between Check Point R81.20 Take 53 and Cisco Meraki. It is a policy-based VPN (IKEv1).

Let's consider this rudimental schema:

1. If I ping the PC (192.168.1.1) from Cisco Meraki (ETH1 192.168.2.254), (by using tcpdump on Check Point) I can see the traffic reaching the Check Point; the PC responds, but I don’t see these replies on Cisco Meraki.

2. If I ping Cisco Meraki (ETH1 192.168.2.254) from the PC (192.168.1.1), I see in the Smart Log that the traffic is encrypted, but, again, it doesn’t reach Cisco Meraki.

Any thought ?

Thank you.

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

PhoneBoy — Wed, 30 Oct 2024 21:11:33 GMT

What does fw monitor say?
Note that fw monitor will show the traffic as it is encrypted/decrypted (though the source/destination IP will change to the tunnel endpoints).

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

cyberluke365 — Wed, 30 Oct 2024 23:12:46 GMT

Hello,

I made two tests while running fw monitor -F "192.168.1.1,0,192.168.2.254,0,0" -F "192.168.2.254,0,192.168.1.1,0,0".

1. PING 192.168.2.254 (from PC1 - 192.168.1.1)

[vs_0][ppak_0] bond1:i[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=60 id=62652 ICMP: type=8 code=0 echo request id=1 seq=814 [vs_0][fw_3] bond1:i[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=60 id=62652 ICMP: type=8 code=0 echo request id=1 seq=814 [vs_0][fw_3] bond1:I[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=60 id=62652 ICMP: type=8 code=0 echo request id=1 seq=814 [vs_0][fw_3] bond1.9:o[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=60 id=62652 ICMP: type=8 code=0 echo request id=1 seq=814

ICMP packets never reach Cisco Meraki.

2. PING PC1 - 192.168.1.1 (from 192.168.2.254)

[vs_0][ppak_0] bond1.9:iD[44]: 192.168.2.254 -> 192.168.1.1 (ICMP) len=84 id=0 ICMP: type=8 code=0 echo request id=10670 seq=0 [vs_0][ppak_0] bond1.9:i[44]: 192.168.2.254 -> 192.168.1.1 (ICMP) len=84 id=0 ICMP: type=8 code=0 echo request id=10670 seq=0 [vs_0][fw_0] bond1.9:i[44]: 192.168.2.254 -> 192.168.1.1 (ICMP) len=84 id=0 ICMP: type=8 code=0 echo request id=10670 seq=0 [vs_0][fw_0] bond1.9:I[44]: 192.168.2.254 -> 192.168.1.1 (ICMP) len=84 id=0 ICMP: type=8 code=0 echo request id=10670 seq=0 [vs_0][fw_0] bond1:o[44]: 192.168.2.254 -> 192.168.1.1 (ICMP) len=84 id=0 ICMP: type=8 code=0 echo request id=10670 seq=0 [vs_0][fw_0] bond1:O[44]: 192.168.2.254 -> 192.168.1.1 (ICMP) len=84 id=0 ICMP: type=8 code=0 echo request id=10670 seq=0 [vs_0][ppak_0] bond1:i[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=84 id=62732 ICMP: type=0 code=0 echo reply id=10670 seq=0 [vs_0][fw_0] bond1:i[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=84 id=62732 ICMP: type=0 code=0 echo reply id=10670 seq=0 [vs_0][fw_0] bond1:I[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=84 id=62732 ICMP: type=0 code=0 echo reply id=10670 seq=0 [vs_0][fw_0] bond1.9:o[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=84 id=62732 ICMP: type=0 code=0 echo reply id=10670 seq=0 [vs_0][fw_0] bond1.9:O[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=84 id=62732 ICMP: type=0 code=0 echo reply id=10670 seq=0 [vs_0][fw_0] bond1.9:Oe[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=84 id=62732 ICMP: type=0 code=0 echo reply id=10670 seq=0

ICMP packets reach PC1, but replies never come back Cisco Meraki.

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

the_rock — Thu, 31 Oct 2024 14:20:04 GMT

So, just for the context, in fw monitor, though Im sure you may know this, but in case you did not...D means decrypted and E means encrypted, but if you say packet never comes back, did you do basic zdebug on CP to see if anything possibly gets dropped?

What do logs show on Meraki end?

On a side note, is this domain or route based? How are enc domains configured? I ask, because IF you have combination of hosts/subnet, then you can set tunnel mgmt tab inside vpn community in smart console as "per gateway". Happy to do remote and assist, if you are allowed to.

Andy

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

cyberluke365 — Fri, 01 Nov 2024 11:13:58 GMT

Hello @the_rock ,
thank you for your useful information. Well, I'll provide more details.

1. Let's start with fw monitor. This is the complete trace for 1 ICMP packet - PING from PC to Cisco Meraki

[vs_0][ppak_0] bond1:i[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=60 id=62769 ICMP: type=8 code=0 echo request id=1 seq=872 [vs_0][fw_1] bond1:i[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=60 id=62769 ICMP: type=8 code=0 echo request id=1 seq=872 [vs_0][fw_1] bond1:I[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=60 id=62769 ICMP: type=8 code=0 echo request id=1 seq=872 [vs_0][fw_1] bond1.9:o[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=60 id=62769 ICMP: type=8 code=0 echo request id=1 seq=872 [vs_0][fw_1] bond1.9:O[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=60 id=62769 ICMP: type=8 code=0 echo request id=1 seq=872 [vs_0][fw_0] bond1.9:Oe[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=60 id=62769 ICMP: type=8 code=0 echo request id=1 seq=872 [vs_0][ppak_0] bond1.9:Oe[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=60 id=62769 ICMP: type=8 code=0 echo request id=1 seq=872

2. This is the complete trace for 1 ICMP packet - PING from Cisco Meraki to Check Point

2.A. It arrives to Check Point:

[vs_0][ppak_0] bond1.9:iD[44]: 192.168.2.254 -> 192.168.1.1 (ICMP) len=84 id=0 ICMP: type=8 code=0 echo request id=10926 seq=0 [vs_0][ppak_0] bond1.9:i[44]: 192.168.2.254 -> 192.168.1.1 (ICMP) len=84 id=0 ICMP: type=8 code=0 echo request id=10926 seq=0 [vs_0][fw_0] bond1.9:i[44]: 192.168.2.254 -> 192.168.1.1 (ICMP) len=84 id=0 ICMP: type=8 code=0 echo request id=10926 seq=0 [vs_0][fw_0] bond1.9:I[44]: 192.168.2.254 -> 192.168.1.1 (ICMP) len=84 id=0 ICMP: type=8 code=0 echo request id=10926 seq=0 [vs_0][fw_0] bond1:o[44]: 192.168.2.254 -> 192.168.1.1 (ICMP) len=84 id=0 ICMP: type=8 code=0 echo request id=10926 seq=0 [vs_0][fw_0] bond1:O[44]: 192.168.2.254 -> 192.168.1.1 (ICMP) len=84 id=0 ICMP: type=8 code=0 echo request id=10926 seq=0

2.B. PC replies:

[vs_0][ppak_0] bond1:i[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=84 id=62773 ICMP: type=0 code=0 echo reply id=10926 seq=0 [vs_0][fw_0] bond1:i[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=84 id=62773 ICMP: type=0 code=0 echo reply id=10926 seq=0 [vs_0][fw_0] bond1:I[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=84 id=62773 ICMP: type=0 code=0 echo reply id=10926 seq=0 [vs_0][fw_0] bond1.9:o[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=84 id=62773 ICMP: type=0 code=0 echo reply id=10926 seq=0 [vs_0][fw_0] bond1.9:O[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=84 id=62773 ICMP: type=0 code=0 echo reply id=10926 seq=0 [vs_0][fw_0] bond1.9:Oe[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=84 id=62773 ICMP: type=0 code=0 echo reply id=10926 seq=0 [vs_0][ppak_0] bond1.9:Oe[44]: 192.168.1.1 -> 192.168.2.254 (ICMP) len=84 id=62773 ICMP: type=0 code=0 echo reply id=10926 seq=0

I looked at the tables reported in "fw monitor" section of R81.20 CLI Reference Guide in order to understand the meaning of "i, I, O,..." (as per your suggestion).

3. Questions:

3.1. I'm not sure what does "Oe" mean; something like Post-Outbound+Pre-Outbound VPN ?
3.2. By the traces (I'm focusing on traffic initiated by PC - point 1 - or reply traffic from PC - point 2) I see the "E" is missing; so the packet isn't encrypted...is it correct ? If it is correct, so why in SmartLog I'm seeing these packets as "Encrypted" (those initiated by PC - point 1):

Now I provide more info about this site-to-site VPN:

This VPN was setup long time ago (it's not brand new); this issue started just few days ago
We have other VPNs configured at the same manner (Check Point- Cisco Meraki) with no issue
No changes were made on Cisco Meraki or Check Point
On Cisco Meraki I see WAN packets exchanged by Cisco Meraki and Check Point (related to site-to-site VPN); (tunnel) packets leaving Cisco Meraki (point 2 above); but no (tunnel) packets from Check Point.

These are site-to-site details:

Encryption Method: IKEv1 (policy-based)

IKE - Phase 1

Encryption Algorithm: AES-128

Data Integrity: SHA1

Diffie-Hellman group: 2 (1024 bit)

Renegotiation: 480 minutes

IPSec - Phase 2

Encryption Algorithm: AES-128

Data Integrity: SHA1

PFS: Off

Renegotiation: 28800 seconds

Check Point

VPN Domain: 5 IP class

Cisco Meraki

VPN Domain: 1 IP class

I have learned that colleagues at Site B have experienced Internet connectivity slowdown issues which led them to open a support ticket with the local ISP. I am concerned that the ISP may have made changes (to their infrastructure) that would explain the described behavior.

However, I would like to be certain that the problem does not lie with Check Point.

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

the_rock — Fri, 01 Nov 2024 11:35:25 GMT

All valid questions @cyberluke365 . So Oe flafg means Post Outbound, encrypted, so it means leaving CP encrypted. What happens to it after, I have no idea.

Andy

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

RS_Daniel — Fri, 01 Nov 2024 13:03:06 GMT

Hello,

Yes, Oe means Outbound before encrypt, check this link:

https://community.checkpoint.com/t5/Security-Gateways/R80-x-Performance-Tuning-and-Debug-Tips-fw-monitor/m-p/41563

It seems to me that firewall is encrypting the packets. You could check if some new NAT rule is being applied to this traffic, that should be reflected on logs. If there is no NAT, i would do a ping from 192.168.1.1 to 192.168.2.254 with size 500 bytes, and check that traffic on external interface, you should filter tcpdump with the peer ip address, esp or nat-t, and greater than 500. The size of the packet is just a way to identify them after encryption, so you could use any size. If traffic does not leave the firewall, check drops with fw ctl zdebug drop. If they leave the firwall, check the meraki device. If you say it was working before it should be ok, but i would double check which encryption domain are being negotiated in phase 2 (vpn tu tlist). Also, as always... should open a case with TAC. hth.

regards

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

the_rock — Fri, 01 Nov 2024 13:44:02 GMT

Also, forgot to ask again, did you make sure what I mentioned in my previous post? If you use combination of hosts/subnets. you should set tunnel mgmt tab in vpn community in smart console as "per gateway" and install policy.

Andy

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

PhoneBoy — Fri, 01 Nov 2024 15:22:04 GMT

When you initiate the ping, do you see the outgoing IPsec traffic leaving your local gateway?
If you do, and given it appears on your end the traffic is being properly encrypted (as evidenced by the Oe packet in fw monitor), the problem is unlikely to be on your end.

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

cyberluke365 — Sat, 02 Nov 2024 12:16:16 GMT

Hello @RS_Daniel,

thank you for sharing the link.

Could you please explain your statement "it seems to me that firewall is encrypting the packets" ? This part of the table at the link you provided:

The latest packet from fw monitor (PING) reports "Oe". Comparing it with table above, if I'm not mistaken, this indicates Pre-Outbound VPN, Outbound before encrypt. So it isn't encrypted yet, as the "OE" state is missing. It seems to me that packet is stuck between the Pre-Outbound VPN and Post-Outbound VPN, correct ? However from SmartLog I see the ICMP packets as encrypted (screenshot I post before). So maybe there is something wrong in the logic I wrote here.

I confirm there is no NAT. The ping -l 500 192.168.2.254 returns Request timeout. While doing it, I run the tcpdump of external interface and this is the capture results filtered by the public IP of Site B:

Legend

*.66 - Public IP of Site B
*.30 - Public IP of Site A

I also opened a case with TAC; I'm waiting for their first contact.

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

the_rock — Sat, 02 Nov 2024 12:19:19 GMT

What it means essentially is that traffic is going through the tunnel, hence the Oe flag in fw monitor.

Andy

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

cyberluke365 — Sat, 02 Nov 2024 12:19:21 GMT

Hello @the_rock,
VPN Domain (on both sites) are subnets, no hosts. VPN Tunnel Sharing is set to One VPN tunnel per subnet pair.

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

the_rock — Sat, 02 Nov 2024 12:21:04 GMT

K, no issues on that front then.

Andy

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

RS_Daniel — Mon, 04 Nov 2024 12:34:45 GMT

Hello,

Yes, it is correct that Oe is before encryption, but if gateway is tagging this packet for encryption it is a good sign, it is normal not to see OE because the IP's change, they are encapsulated and you can only see the public IP's instead (that is why the tcpdump trick helps us to check if the packet is leaving). On the other hand, on the capture you provided, how long are ESP packets? if they are 548 bits or something like that, that is your ping being encrypted and sent to the remote peer, that would mean the firewall is doing its job.

Regards

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

cyberluke365 — Mon, 04 Nov 2024 13:05:06 GMT

Hello @RS_Daniel, @the_rock, @PhoneBoy

This morning, the site-to-site VPN started working again. I didn’t change anything in the configuration, and the investigation conducted (thank you for your useful information) confirms that the issue was not attributable to Check Point or Cisco Meraki but likely 'in the middle' (that is, within the Internet connectivity).

Thank you.

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

the_rock — Mon, 04 Nov 2024 13:10:37 GMT

Thats great to know!

Andy

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

PhoneBoy — Mon, 04 Nov 2024 18:25:10 GMT

When the packet hits OE, it will be fully encrypted (source of local gateway, destination of remote gateway).
fw monitor will only show it if the filter includes those IP addresses (otherwise, it won't show it).

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

Matlu — Sat, 31 May 2025 14:42:59 GMT

Hello
If you have an S2S VPN that is in UP, both Phase 1 and Phase 2, but your traffic is not being 'Encrypted/Decrypted' and you don't see it going through the VPN TUNNEL you have built either, this may be some configuration error in the Phase 2 part of the VPN?
I use policy based VPN, in R81.20 with JHF 98
Can I use some diagnostic command to find out why my traffic is not going through the VPN tunnel?
It's weird, because the VPN is up but there is no traffic through the tunnel.

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

the_rock — Sat, 31 May 2025 15:07:52 GMT

Hey bro,

You can always do fw monitor. Fwiw, try vpn accel off as well as a test.

Andy

Re: Checkpoint Site to Site VPN, Tunnel is UP, but traffic doesn't arrive to destination

PhoneBoy — Mon, 02 Jun 2025 13:52:34 GMT

You could probably get that information through debugging the VPN module: https://sc1.checkpoint.com/documents/R81.20/WebAdminGuides/EN/CP_R81.20_PerformanceTuning_AdminGuide/Content/Topics-PTG/Kernel-Debug/Module-VPN.htm
Having said that have you confirmed there are routes for the relevant networks that point to the VPN tunnel?