topic Re: 'TCP packet out of state' drops in Firewall and Security Management

'TCP packet out of state' drops

arcotangente — Fri, 15 May 2020 11:13:46 GMT

Hi guys,

We are troubleshooting an issue and see many HTTPS packets dropped with the following message in the logs:

'TCP packet out of state -First packet isn't SYN'

I've tried to disable this protection for one specific source, so open Inspection settings, and added an Exception for this specific source IP (all protections, profiles and destinations)

However I still see packets being dropped with the same message in the logs.

Is there a way to bypass an specific source or destination of this protection?

Thanks

Re: 'TCP packet out of state' drops

Timothy_Hall — Wed, 20 May 2020 12:52:51 GMT

What TCP flags (RST, FIN, ACK, etc.) are you seeing on the packets dropped as out of state? If they are RST or FIN the connection is already dead so you can probably ignore those. If the flags on the dropped packets are SYN and ACK (or perhaps just ACK), that may indicate asymmetric routing going around the firewall. If the flags on the dropped packet are some combo of only ACK/PSH/URG usually that means the connection was timed out by the firewall, in that case you can try increasing the service timeout for HTTPS on the Advanced screen of the matching HTTPS service.

Re: 'TCP packet out of state' drops

arcotangente — Fri, 15 May 2020 13:33:00 GMT

Thanks Timothy,

The flags are 'PUSH-ACK'

Re: 'TCP packet out of state' drops

Timothy_Hall — Fri, 15 May 2020 14:25:39 GMT

Try increasing the timeout for the HTTPS service on its Advanced screen, and make sure you modify the correct HTTPS/port 443 service that is actually matching the problematic traffic as there may be several defined.

Re: 'TCP packet out of state' drops

arcotangente — Wed, 20 May 2020 10:19:54 GMT

Hi,

Finally the issue got fixed after a reboot of the secondary node, while troubleshooting another issue. Difficult to understand what happened

Thanks anyway!

Re: 'TCP packet out of state' drops

anettekr — Wed, 23 Aug 2023 14:47:24 GMT

Hello, i know this is an old post but hoping you might be able to help.

i am having some issues pin pointing connection resets for a payment application. trying to find out if the firewall is at fault so something else is along the path.

i'm seeing First Packet isn't SYN being dropped by the firewall with the flag -FIN-PUSH-ACK. the timings of these drops don't match exactly with the connection errors seen for the payment application but there are some of these drops around the same time as connection problems. we also see them regularly throughout the day.

not done any work to resolve just yet as not sure what the best approach would be seen various 'solutions': change TCP end Timeout, exclude IPs from SecureXL. disable HTTPsi, its enabled but connection is already excluded as its payment related. Failover firewalls.

would you have any suggestions based on the flag we see, FIN-PUSH-ACK

Many thanks,

Anette

Re: 'TCP packet out of state' drops

Timothy_Hall — Wed, 23 Aug 2023 16:51:29 GMT

Unlikely that the FIN-PUSH-ACK drops are related to your suspected RST issue, as the presence of those flags reveals that the host already believes the connection is over and is trying to end it gracefully. That connection no longer being present in the firewall's state table and the drop occurring shouldn't really affect that. I suppose the payment app may hang waiting for the connection to gracefully end before it is able to launch a new connection and continue normal operation, and that may look like an interruption/outage.

It is also possible that the long-running payment application connection is getting silently timed out by the firewall, the application realizes this and tries to close it gracefully but gets stuck. If you suspect this is the case you could try increasing the timeout for the relevant TCP service it is using, or try changing fw_rst_expired_conn from 0 to 1. This will cause a RST to be sent immediately upon expiration, which may help the host figure out the connection is gone and to launch a new one immediately.

Re: 'TCP packet out of state' drops

anettekr — Thu, 24 Aug 2023 08:36:40 GMT

Thank you for the reply, much appreciated ill do some digging into what you've mentioned.

i am reluctant to alter the service as its HTTPs and not sure if it would end up impacting other connections. and will see what the impact could be for changing the fw_rst_expired_conn setting.