Routing host to server behind VPN tunnel

[NorthernNetGuy]

Running into a routing issue that I could use some advice on.

We have a VPN Community set up with a vendor, creating a tunnel between some of our servers and their servers. This tunnel is functioning fine for the server communications. Our VPN domain is the private subnet the required servers are in, and their VPN Domain is a range of public IP addresses. We have 1 public IP address we use for all our communications.

The Vendor hosts a website that we need our clients(non tunneled) to be able to access, but the IP is one of the ones in their VPN domain for the tunnel we have established. Our clients can't reach this site.

I'm suspecting this is because we use the same public IP assigned to eth1 for our client traffic and the VPN, 

What's the best way to get this to work?

[PhoneBoy]

So your firewall has the IP your clients need to connect to assigned to one of its NICs?
Yeah, you'll need to change that somehow: either change the IP address the end users connect to OR change the IP assigned on your gateway.
That might mean a NAT rule is required so users can connect to that different IP.

[NorthernNetGuy]

Heres a Diagram to help better Explain.

The client PC can't reach the website, When attempting to tracert to the vendor public site, it doesnt get routed thru the firewalls default route out to the internet. I'm assuming this is because it wants to pass it thru the tunnel, because the destinaion ip is in the 3rd party vpn domain, but the client is not in the local VPN domain.

[AaronCP]

If the client network is not in the VPN encryption domain, I wouldn't expect it to enter the VPN tunnel to reach the destination host. If you run ip route get x.x.x.x (destination host) on your edge firewall, does it return your default route? Also, if you run an FW Monitor on the gateway on a client IP attempting to reach the destination, do you see the traffic get NAT'd and leave via your external interface? In the capture, does it show O or Oe?

[NorthernNetGuy]

the ip route get does return my expected default route, so that's good.

this is what i'm getting from the FW Monitor output. should I be using some specific flags or turning of accell for this capture?

fw monitor -e "host(10.X.X.X) and host(209.X.X.X), accept;"

[vs_0][fw_1] eth1-02:i[44]: 10.X.X.X -> 209.X.X.X (TCP) len=52 id=938 TCP: 15798 -> 443 .S.... seq=28e914ed ack=00000000
[vs_0][fw_1] eth1-02:i[44]: 10.X.X.X -> 209.X.X.X (TCP) len=52 id=941 TCP: 15799 -> 443 .S.... seq=f9f6952e ack=00000000
[vs_0][fw_3] eth1-02:i[44]: 10.X.X.X -> 209.X.X.X (TCP) len=52 id=944 TCP: 15800 -> 443 .S.... seq=84347a49 ack=00000000

[AaronCP]

Try fw monitor -F "10.x.x.x,0,209.x.x.x,0,0" - you don't need to disable SecureXL with the -F flag.

[NorthernNetGuy]

I see a couple packets with the O flag, looks like NAT'd packets, as it shows the firewall public IP 204.x.x.x:

It's not appearing at the rate i'd expect

[vs_0][ppak_0] eth1-02:i[44]: 10.x.x.x -> 209.x.x.x (TCP) len=52 id=63617
TCP: 21306 -> 443 .S.... seq=8b0adfea ack=00000000
[vs_0][fw_0] eth1-02:i[44]: 10.x.x.x -> 209.x.x.x (TCP) len=52 id=63617
TCP: 21306 -> 443 .S.... seq=8b0adfea ack=00000000
[vs_0][ppak_0] eth1-02:i[44]: 10.x.x.x -> 209.x.x.x (TCP) len=52 id=63617
TCP: 21306 -> 443 .S.... seq=8b0adfea ack=00000000
[vs_0][fw_0] eth1-01:I[44]: 209.x.x.x -> 10.x.x.x (TCP) len=52 id=58262
TCP: 443 -> 21306 .S..A. seq=d5b05b63 ack=8b0adfeb
[vs_0][fw_0] eth1-02:o[44]: 209.x.x.x -> 10.x.x.x (TCP) len=52 id=58262
TCP: 443 -> 21306 .S..A. seq=d5b05b63 ack=8b0adfeb
[vs_0][fw_0] eth1-02:O[44]: 209.x.x.x -> 10.x.x.x (TCP) len=52 id=58262
TCP: 443 -> 21306 .S..A. seq=d5b05b63 ack=8b0adfeb
[vs_0][ppak_0] eth1-02:i[40]: 10.x.x.x -> 209.x.x.x (TCP) len=40 id=63618
TCP: 21306 -> 443 ....A. seq=8b0adfeb ack=d5b05b64
[vs_0][fw_0] eth1-02:I[44]: 10.x.x.x -> 209.x.x.x (TCP) len=52 id=11363
TCP: 21306 -> 443 .S.... seq=8b4adfea ack=00000000
[vs_0][fw_0] eth1-01:o[44]: 10.x.x.x -> 209.x.x.x (TCP) len=52 id=11363
TCP: 21306 -> 443 .S.... seq=8b4adfea ack=00000000
[vs_0][fw_0] eth1-01:O[44]: 204.x.x.x -> 209.x.x.x (TCP) len=52 id=11363
TCP: 20388 -> 443 .S.... seq=8b4adfea ack=00000000
[vs_0][fw_0] eth1-01:I[40]: 209.x.x.x -> 10.x.x.x (TCP) len=40 id=58396
TCP: 443 -> 21306 ....A. seq=d5b05b64 ack=8b0adfeb
[vs_0][fw_0] eth1-02:o[40]: 209.x.x.x -> 10.x.x.x (TCP) len=40 id=58396
TCP: 443 -> 21306 ....A. seq=d5b05b64 ack=8b0adfeb
[vs_0][fw_0] eth1-02:O[40]: 209.x.x.x -> 10.x.x.x (TCP) len=40 id=58396
TCP: 443 -> 21306 ....A. seq=d5b05b64 ack=8b0adfeb
[vs_0][ppak_0] eth1-02:i[44]: 10.x.x.x -> 209.x.x.x (TCP) len=557 id=63619
TCP: 21306 -> 443 ...PA. seq=8b0adfeb ack=d5b05b64

[PhoneBoy]

That looks like a successful connection with the remote end.
What am I missing here?

[NorthernNetGuy]

I finally took a packet capture on my local machine, I'm receiving lots TCP Resets from them, so It looks like my routing should be fine then. I'm not sure why they're sending RST yet. An odd scenario, but I tried connecting to the site on an old 2008 server, and it's able to successfully connect, no Resets.  It's in a different subnet, but basically the same routing.

I'll do some further investigation and report back. Any suggestions are appreciated as well!

[AaronCP]

Is your old server being NAT'd behind the same public IP as your client networks? And the old server IP is definitely not in your VPN domain?

[NorthernNetGuy]

the old server is being NAT'd behind the same public IP as the client networks, and it is not in the VPN domain

[Chris_Atkinson]

Have you reviewed the SK and scenario I referenced above?

[NorthernNetGuy]

Hi Chris, Looks like that SK wasn't relevant to the solution.
I do have a secondary related issue. My Mobile Access VPN clients can't reach the website either, with the logs showing 
"Connection terminated before the Security Gateway was able to make a decision: Insufficient data passed.
To learn more see sk113479."

I've reviewed this SK but i'm not seeing a clear solution yet.

[PhoneBoy]

At a high level, you will get this message in the logs when:

• One or more rules that allow the traffic use an application in the service column 
• Not enough data was passed on the connection to determine whether or not the traffic matches one of those applications

The traffic is ultimately being allowed, and it's by design.
And it's also not a problem specific to Check Point either: https://phoneboy.org/2016/12/14/which-comes-first-the-ports-or-the-application-id/ 

[NorthernNetGuy]

That's what I've read, however the traffic doesn't appear to be allowed. While troubleshooting, I did a tracert from one of the mobile access vpn clients. The traffic is dropped before being sent to the gateways default route. the gateway itself can successfully tracert to the website. SSL inspection is bypassed for these connections as well.

[PhoneBoy]

What does fw ctl zdebug drop show?

[NorthernNetGuy]

It looks like this is actually an SSL inspection issue. I just realized the server is in a group object for ssl inspection bypass.

I had the sites URL and IP bypassed for all users, but looks like there are other resources causing issues. After adding the client PC to bypass all outbound traffic, the site became reachable.
I'm not seeing anything besides the sites IP and URL being referenced, not sure what else I would need to bypass for it as the destination.

[Chris_Atkinson]

Sounds like sk108600 scenario 3?

(Scenario 3 - Implied inclusion of Check Point Security Gateway's / 3rd party VPN Peer's interfaces)