VPN community routing not working

[isuckatthis]

Hello smarter than I people!

I have a Checkpoint to Checkpoint VPN. We're using a community. The traffic gets to the Checkpoint FW VPN concentrator and is not routed across the VPN.

How do I troubleshoot this?

The encryption domain on the remote side shows the correct subnets.

I've attached images showing the community, logs showing the traffic is not encrypted, a traceroute showing the FW is sending the traffic back to the router then the router back to the FW then the FW back to the router until the end of time (or TTL expires).

I have no idea how to troubleshoot this. Help! 😞 

[isuckatthis]

Then you *definitely* can't do this.  You have 10.0.0.0/8 out that VTI to the Cisco gateway.  However, you are also trying to do a domain-based VPN to your other gateway, also with networks within the 10.0.0.0/8 supernet.  Domain-based VPNs take priority over route-based VPNs (the VTI).  I want 10.59.0.0/16 to go on the domain based VPN and everything else in 10.0.0.0/8 to go through the VTI. That's not possible? You are risking breaking the VTI to the Cisco gateway.  The reason is hasn't broken yet is because the hub hasn't seen packets to it (yet).

You *CAN* do route-based VTIs on all of the gateways and control packet pathing in the route tables.  This would be your safest option.  Be very careful and very judicious about domain-based VPNs with route-based VPNs.  Maybe we have to re-design and scrap the VPN domains, sounds like that's your recommendation.

Your first (and most significant) issue is the routing and networking on your hub gateway.  Having an interface configured like eth4 (the /29 subnet) in addition to a larger network route (10.59.0.0/16) out the same interface will also cause weird effects making debugging/troubleshooting difficult.  If you want eth4 to be a transit VLAN/subnet, then make it a real transit on its own transit subnet. It is a transit link. I don't have an IP in the 10.59.0.0/16 configured on eth4. Taking the italicized above at face value, I am hearing you say, "you cannot have 10.0.0.1/29 configured on your firewall and 10.0.0.2/29 configured on your router with a route on the FW of 10.59.0.0/16 next hop 10.0.0.2" that doesn't make sense, that's how routing works. What am I missing here?

With the VPN domains configured like you have them, your logs will also be showing various VPN errors. No VPN errors at all, unless I'm looking at the wrong logs.

[Lesley]

You have overlap in the routing aswell the 10/8 is routed to the ASA routed based tunnel. routed based tunnel have prio over domain based tunnel

[isuckatthis]

Are you saying that the decision making process is this:

Delineated routing tables

Section 1 (route based VPN) of the table is reviewed first. If a route is found that matches the destination, it's automatically used.
Section 2 (policy based VPN) of the table is reviewed ONLY if there is NO route found in section 1, including no 0/0

Do I understand correctly?

[the_rock]

I think I now see what your issue is. Please change in community vpn tunnel sharing options per subnet, NOT per gateway, as per gateway option will do universal 0.0.0.0/0 subnet.

Andy

[isuckatthis]

Which one? I have many issues. 😂
Testing now.............

No go. Changed to per subnet and the traffic is still not tunneled.

[Lesley]

The routers sends back the traffic because it is being send out unencrypted. 

Routers sees: dst: 10.59.78.2 it would route this back to the fw. It should see the public IP of the remote peer.

Between the public peer IPs you should see ike 500 and ESP traffic. No RFC1918 traffic. Router should only see encrtyped traffic.

So focus on why it is not encrypted. You have checked the remote side encryption domain but also check local side 10.59.78.2 I suspect this host / network is not in local enc domain. 

[isuckatthis]

The source IP of the traffic behind the hub is in the hubs encryption domain object.

[isuckatthis]

Agreed, it's not tunneling to the spoke. I don't know why. All I know is that the encryption domain tells the FW what to tunnel. Am I wrong? Are there other pieces that I'm missing?

This is my encryption domains. I've applied them on the side where that network lives. For example, my remote site hosts the network 10.59.78.0/24. Therefore, I've created a network object for that subnet and used that object (ThompsonTesting...) in the VPN domain on the REMOTE FW. I assume that is telling the hub "go to the remote for 10.59.78.0"

Vice versa, I've created three objects to be used by the hub side VPN domain. 10.59.40.69 (a loop back on a router which I'm using to test) then two additional networks that are prod. 

I don't think so, but, should I be putting the opposite sides networks inside the VPN domain for the local host?

[the_rock]

What time zone are you in? Happy to do remote later if you are free or tomorrow morning, let me know. Im in EST.

Andy

[the_rock]

If you see this message before 9 am EST, be free to send me a DM, we can do remote (If you are allowed and willing to do it). I would really like to have a look at the config, because I have a gut feeling something trivial is missing here.

Andy

[isuckatthis]

Hey Andy, I really appreciate the offer! 

I saw Duane post last night but now I think it's been deleted. I understood from it that the VPN Domain does NOT add a top secret hidden route, I will need to add a route and ensure it's egress is the external interface on this firewall.

I'm about to test that now. I'd be happy to have a remote session if you're interested. 

Tested and now the traffic is being dropped for spoofing. We have anti-spoofing enabled on the inside interface and it's using the 10.59.0.0/16. I'm assuming this is the FW logic:
1 packet comes in and is to be routed out the external interface based on my /24 route

2 FW says "wait a minute, 10.59.0.0/16 is all behind the inside interface based on the anti-spoofing list you've configured in me"

3 FW drops the traffic even though I am also telling it "route out your external interface for 10.59.78.0/24"

So now it seems that I have to make 255 individual /24's and add them to an anti spoofing list and remove the 10.59.0.0/16, that seems, inefficient. I don't have any anti-spoofing list on the external interface. I assume that if it's labeled as "external" it means anything NOT behind any other interfaces.

[Duane_Toler]

Correct, domain-based VPNs do not add routes to the routing table (unless you have RIM configured, but that's not this).

My post from last night is still here in the thread, too.

[the_rock]

What time zone are you in? I can do say 4 pm est.

Andy

[isuckatthis · Accepted solution]

Thank you all for helping with this, the tunnel is UP!

My problem was a slight domino effect. 

1) I understood the VPN Domains incorrectly
2) As was mentioned by a couple folks, I cannot have overlap in the VPN Domain. The resolution for this is pretty cool and I got to learn something new! We made a Network Group with Exclusion! The main object included 10.59.0.0/16 and the except value was 10.59.78.0/24, works perfectly!

Thank you all for your time invested in helping me understand this. The best part about it was learning something new that I can apply in the future!

[the_rock]

Thats EXCELLENT, good to keep in mind! Great job.

ANyd