First packet isn't syn

[tlloyd22]

Hey everyone.  I have a new CPGW R81.10 and I have one workstation that's dropping traffic 3 to 4 times a second with the following issue:

TCP packet out of state: First packet isn't SYN

TCP Flags: RST-ACK and FIN-PUSH-ACK

Can this be ignored?  I can't say I'm seeing a perf problem.  Or, should/how can it be fixed?  Thanks all!

[Timothy_Hall · Accepted solution]

Generally you can ignore FIN and RST packets that are dropped out of state unless they are conclusively linked to a specific problem.  This is typically caused by the connection not being closed gracefully by one side or the other, see my post here:

https://community.checkpoint.com/t5/General-Topics/First-packet-isn-t-SYN/m-p/7027/highlight/true#M7...

[George_Ellis]

If you do see a lot of these FIN or RSTs around home built applications, especially after they did a code release, let the dev team know.  They may not be ending a connection gracefully and might have missed a commit on their data connections.  Won't happen often, but you never know...

[maad-pul]

Hi,

Did you find anyting related to this issue? I have also seen those packets dropped after upgrading to R81.10 TAKE30.
I haven´t seen any drops related to TCP Flags: RST-ACK and FIN-PUSH-ACK before upgrade R80.40. 

My problem is that I have performance issues related to flows where I see errors in logs.

I have disabled HTTPS Inspection which seems to solve the issue. 
 

[tlloyd22]

After seeing Tim Hall's post and reading through, I chose to ignore the errors since I didn't see a performance issue.  Sorry I'm not more helpful...good luck!

[the_rock]

Are you seeing any performance impact, dropped traffic? If so, then I would be concerned...if not, then I would not worry much.

[maad-pul]

Hi,

Yes! This weekend we upgraded to lastest R81.10 GA,  I´m having users reported "slow web browsning" today, I found in logs a lot of dropped packet related to tcp/443, which haven¨t been there before. 

TCP packet out of state: First packet isn't SYN
TCP Flags: RST-ACK and FIN-PUSH-ACK

I quick fix, just to now whats goding on was to disable HTTPS Inspection for that VS. Logs dissapeard and users reported good web browsing performance after that. The Logs are from the FW-blade and not HTTPS Inspection.

[the_rock]

You may need to debug wstlsd process for https inspection, when its enabled.

[CheckPointerXL]

hello, did you fix it?

[maad-pul]

Hi 

Yes, patched an the problem was solved. According to TAC PRJ-30820 was the issue. 

[CheckPointerXL]

i'm happy for you.

I don't think that's my case, PRJ-30820 is on T38 and i faced the problem on T55

[skandshus]

I am seeing a LOT of those too in my ubiquiti unifi controller where connection status/heartbeats then gets time out and throwing me an Alert of a device disconnecting even though it’s working fine. It’s so annoying.. never had that issue before checkpoint unfortunately 😞

[the_rock]

I would open TAC case for it to have them verify, specially given the fact it causes traffic issues.

[maad-pul]

Hi!

I will do that, I just need to verify Hyperthreading (SMT/HT) enabling i BIOS for the HPE-server.  I don´t know if thats setting was enabled in last update to R80.40, but support seems to be removed. 

See https://community.checkpoint.com/t5/Security-Gateways/Attention-HyperThreading-SMT-support-for-Open-....

[Pedro_Sentinela]

hello, I'm having the same problem as you and I'm losing a lot of packages that are destroying the performance. Could you let me know if there was a solution to your problem?

[skandshus]

mine isnt fixed.

from my experience it doesnt look like Check point have a FIX for it actually 😞

[the_rock]

Well, first packet isnt syn error is not something CP would have a fix for, per se, its usually routing issue, from my experience.

[skandshus]

Routing issue in what sense?

[Chris_Atkinson]

Indeed, need to ensure the symptoms are identical i.e. in no particular order:

- Asymmetric routing (both direction/sides of the flow need to traverse the firewall)

- Aggressive aging (due to resource issue or other transient event)

- App issue / TCP half-closed (sk11088 / sk137672)

- HTTPS inspection (sk118415)

[skandshus]

i've had a TAC case but nothing resolved so far.

Aggresive aging or APp issue?<< what do you think here?

[George_Ellis]

Zombie alert - Adding to your list Chris:
- Missing keep alive by the application. (and remember Oracle must be restarted if you turn on keep alive! - that took a few months until an Oracle support engineer working for IBM said, "did you restart the instance?"  The protocol stack needs to be rebuilt to add keep alive to the virtual NIC)

- Device in the middle has short keep alive timer (load balancer had a 5 minute keep-alive).

General tip for asymmetric routing, see which interfaces are used outbound and inbound.  You may see eth1 -> dst -> eth2 (s_port is from dst port, d_port matches org src port - you can use matches such as "(s_port=52212 or s_port=443) and (d_port=52212 or d_port=443)" when you identify such a packet to isolate a single piece of traffic.

[Ravi_cp]

I'm facing similar kind of issue after upgrade and pushing policy on standby gateway .we are not able to access the web UI/ssh of gateway.

While further checking in log  found lot of  TCP out of order packet drop for reverse traffic,  for testing purpose added gateway in expectation, than it starting working .  Observers gateway for 2 min and found gateway hang and not responding , so again started new session of ssh but response is too slow again checked resource and found utilisation is fair enough .

So I drop the plan of upgrade and rollback the change .

Upgrade from R80.30 to R81 latest jumbo hotfix.

[_Val_]

Are you sure you do not have an asymmetric routing situation with that GW?

[Ravi_cp]

After rollback it working , is upgrade change the topology and anti spoofing setting .

I used snapshot for rollback 

[_Val_]

This does not answer my original question. It might be that some settings after the upgrade triggered the asymmetric routing. 

[Ravi_cp]

I need to check the routing after upgrade 

[the_rock]

Just do ip r g 8.8.8.8 or ip r g x.x.x.x, where x.x.x.x is any IP you would usually test for routing checks. That would tell you for sure if something looks different.

Andy

[maad-pul]

PRJ-30820 was my issue. 

[Jonne_Hannon]

It appears that in my case, the connection is being expired after 15 seconds when the firewall sees the DEST-FIN.  The client (Chrome/Edge)  continues to send Keep-Alive packets for about 5 minutes, which are all dropped out of state.  The client then sends several FIN-ACK packets, which are also dropped out of state.  This is what the connection looks like before DST-FIN:

<00000000, 0a7a1550, 0000d333, ac1c2015, 00000050, 00000006; 0001c001, 40044080, 00000038, 000001cf, 00000000, 6387085d, 00000008, 13e3b1d3, d7e5c551, 00000003, ffffffff, ffffffff, ffffffff, 0000e800, 00000000, 80000000, 00000000, 00000000, ac442808, 00007f8e, 00000000, 02101801, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000; 3603/3615>

and after the DEST-FIN

<00000000, 0a7a1550, 0000d330, ac1c2015, 00000050, 00000006; 0001e001, 40044080, 00000038, 000001cf, 00000000, 6387085d, 00000000, 13e3b1d3, d7e5c551, 00000003, ffffffff, ffffffff, ffffffff, 0000e800, 00000000, 80000000, 00000000, 00000000, b8fd9088, 00007f8e, 00000000, 06ce0001, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000; 14/15>

After the server sends the FIN-ACK to the client, the connection is updated with this 15 second timeout.  The client (Chrome/Edge) sends the first Keep-Alive packets after 45 seconds and several more Keep-Alive packets at 45 second intervals, These Keep-Alive packets are all dropped out of state, as well as subsequent FIN-ACK packets.  I'm not sure where the 15 second timeout is coming from; the TCP end session timeout is 5 seconds.  Disabling SecureXL resolves the issue.  I have a case open, but not getting anywhere quickly unfortunately.  I'm not sure if this is impacting user experience, other than causing Chrome and Edge to use more sockets than necessary and generating lots of unnecessary logs.  It seems to be that there is an issue with SecureXL expiring half-closed connections too early and would like to get to the bottom of it

[skandshus]

Nice trouble shooting right there 👍