Route based failover causes "First packet isn't SYN" drops for existing connections

[JoeBandura]

Hey all,

I have a some questions about routing traffic over a VPN. I have two different sites that are both connected via fiber. As an alternate path in the event the fiber goes down, I have configured a route based VPN between the two sites gateways. OSPF is configured and this all works as expected. However, when the fiber connection is dropped and routing updates to use the VTI/VPN, existing connects are dropped by the gateways with "First packet isn't SYN" messages.

I assume this is related to the stateful inspection of the gateways and since those connections don't exist in the connections table, they're dropped.  Wire mode is enabled for the VPN, and honestly I would have expected this to take care of it, but it does not. Turning off stateful inspection isn't that straight forward, because it appears to be an all or nothing for everything, limited to individual gateways, or more fine grained via editing the INSPECT code with sk11088.

Right now, it looks like sk11088 is the way to go, and I can just filter everything when the source and destination is both our internal subnets. However, I am worried this may apply to external connections depending on when this is applied, either pre or post NAT, since that's not mentioned in the sk.

So, my questions are:

1. Am I missing something here? It seems like this would be a normal use case for route based VPNs and directing existing traffic over them would be expected. As mentioned, I thought wire mode would take care of it, but it's enabled and does not fix the problem. Is there something else that needs enabled?

2. If modifying the INSPECT code via sk11088 is the route to go, then would excluding connections when the src and dst are always internal networks be an acceptable solution? If yes, how would this affect connections that are external when NAT is applied?

Thanks for the help.

[Gennady]

Hi!

I assume that I have missed something here:
"existing connects are dropped by the gateways with "First packet isn't SYN" messages" and "hose connections don't exist in the connections table, they're dropped". These two statements are in contradiction to each other.

Would you please elaborate more about the connectivity?

As far as I understand normal connection looks like this:
Source -> Firewall A -> fiber -> Firewall B -> Destination

If fiber breaks, then:
Source -> Firewall A -> VPN -> Firewall B -> Destination

Is it a correct assumption?

It would be beneficial to take a look at full connections table entry (connection in HEX + simlinks in HEX) for a connection accepted via Fiber path and then the same connection re-established via VPN path.

[JoeBandura]

To help with the flow a bit, this is what it looks like. I should have included this in the OP. Sorry 🙂

Normally:
Server (src/dst) <-> Router <-> fiber <-> Router <-> Server (dst/src)

If fiber breaks:
Server (src/dst) <-> Router <-> Firewall A <-> VPN <-> Firewall B <-> Router <-> Server (dst/src)

[Gennady]

Well... I don't see the same Firewalls on the flow 1. The routers are connected via the fiber.
On the flow 2 I do see the firewalls.

This way the connections are all new for the firewalls if fiber breaks. This is why you have "First packet isn't SYN" drops.

You need to either add the same firewall in the Flow 1 or remove those from Flow 2 (terminate VPNs on routers). It looks like design problem for now.

[Duane_Toler · Accepted solution]

I do the same with another customer, except with BGP and EIGRP.  It's the same topology you have, tho.  In short, yes, this is the expected and correct behavior when the routing paths change and the original packets don't traverse the firewalls.  Sessions need to restart; nothing (sane) you can do about it.  It's how good firewalls are supposed to work.

If your user population is complaining, then you'll want to reframe the situation to them as: "Yes, this is a one time inconvenience, but you will have restored connectivity to Site B. The alternative is significantly worse, and you wouldn't want that, either."

[JoeBandura]

This ended up being the conclusion I have come to. I guess I just expected the firewalls to act more like a router in this situation since the documentation led me to believe as much with Wire Mode on the VPN and using VTI's for a route based VPN.

No user complaints since this situation is generally rare. And, since connectivity is basically imminently restored when failing over to the VPN, it meets my needs.

Thanks for confirming what I already suspected. 

[Duane_Toler]

Glad to help! If you don't mind, please be sure to click "Accept as Solution" to any posts you think answered your question.