First packet isn't SYN drop on one site only

[Ok1]

Hello Check Point Community!

I’ve encountered the following issue. We have two Maestro sites in two data centers. One site has 1 gateway (there were 2, but currently not added), and the other has 2 gateways. Currently, the setup is working on the first site, which has 1 gateway. Now, we want to add that second gateway to the first site and, accordingly, have redirected traffic to the second site (the one with 2 gateways) via manual failover. However, we’ve run into an issue where, after switching to the second site, we are seeing drop logs saying "First packet isn't SYN" with the TCP push-ACK flag. A custom TCP service is used, and connection synchronization is enabled. Timeout is the default 3600 seconds, where we see no discrepancies in fw ctl conntab output.

We understand this issue might be related to an asymmetric network, but this wasn't a problem before one of the gateways in site 1 had an issue and had to be removed.

Could this issue be caused by the difference in the number of Security Gateway members on the sites?

[the_rock]

Im not Maestro expert by any means (not even close), but I can tell you having seen that message probably more than 200 times, 90% of the time it has to do with routing, as it indicates 3 way handshake is not completing properly.

Andy

[kamilazat]

True and resolvable when it is simply about routing. What about the other 10%? 🙂

It looks like their issue appeared 'out of the blue' after a failover. I remember reading in some SK that sometimes some crashes or failovers can result in deletion of 'some' routes. Though I've no clue how that's even possible.

[the_rock]

I cant say for sure if that could happen, as I had never seen it myself 🙂

Andy

[hugothebas]

What kind of traffic is this? VPN? Input Connections (those which destination is the gateway itself)?

As @the_rock said, That is very common in Maestro environments, it is probably related to the distribution algorithm, so I would start by checking if the distribution configuration is the same on both sites (show distribution configuration) and also if l4 is enabled or not (show distribution l4-mode).

Thebas.

[emmap]

Do you see these for only a few minutes after failover or are they constantly occurring when running on site 2 for a while?

Is there a lot or just a few?

Is the connectivity associated with the drops impacted?

What's your distribution configuration for the interfaces involved? (ingress and would-be egress if they were accepted)

Do you have L4 distribution enabled?

[Ok1]

Thank you @hugothebas  Here is the output:

Host-ch01-02> show distribution configuration
Distribution Mode: auto-topology (per-port)
Host-ch01-02> show distribution l4-mode
L4 Distribution: Enabled

These settings were not touched (left as default). I remember @emmap  recommending disabling L4-mode if we don't really need it. Though I'm not sure if this would result in TCP out of state issues.

@emmap  The issue happens whenever we switch to the other site. Let me share a screenshot from the logs.

Thank you for your help.

[Lloyd_Braun]

I had some issues with L4 mode and fragmented packets. I believe it was because fragmented packets did not have full l4 header with the port numbers so L4 distribution algorithm could not distribute the fragments properly. I am open to correction on that but pretty sure that is what was breaking it.

[hugothebas]

Enabling or disabling L4 mode depends a lot on the kind of traffic you have, I would say, if you have a lot of NAT traffic, for example if your entire organization are NATted behind this Security Group, then L4 would be beneficial for you, if you have dynamic routing, you should not use L4. VPN traffic also should need another tweaking, since you have a "Per-Port" mode, maybe config each port accordingly to its topology.

Each case is a distinct case. You need to map your traffic and see if you should leave it enabled or not.

But I agree with @Lloyd_Braun that L4 could be causing the issue you are facing, you just need to be sure before disabling.

If you didn't do it yet, I'd recommend you open a TAC case.

Best regards.

[emmap]

Disabling l4 mode is a good test. If the traffic is passing two 'internal' or two 'external' interfaces then that could also indicate a correction issue. Is this a perimeter gateway / lots of logs or more of an east/west internal only situation?