'TCP packet out of state' drops

[arcotangente]

Hi guys,

We are troubleshooting an issue and see many HTTPS packets dropped with the following message in the logs:

'TCP packet out of state -First packet isn't SYN'

I've tried to disable this protection for one specific source, so open Inspection settings, and added an Exception for this specific source IP (all protections, profiles and destinations)

However I still see packets being dropped with the same message in the logs. 

Is there a way to bypass an specific source or destination of this protection? 

Thanks

[Timothy_Hall]

What TCP flags (RST, FIN, ACK, etc.) are you seeing on the packets dropped as out of state?  If they are RST or FIN the connection is already dead so you can probably ignore those.  If the flags on the dropped packets are SYN and ACK (or perhaps just ACK), that may indicate asymmetric routing going around the firewall.  If the flags on the dropped packet are some combo of only ACK/PSH/URG usually that means the connection was timed out by the firewall, in that case you can try increasing the service timeout for HTTPS on the Advanced screen of the matching HTTPS service.

[arcotangente]

Thanks Timothy,

The flags are 'PUSH-ACK'

BR

[Timothy_Hall]

Try increasing the timeout for the HTTPS service on its Advanced screen, and make sure you modify the correct HTTPS/port 443 service that is actually matching the problematic traffic as there may be several defined.

[arcotangente]

Hi,

Finally the issue got fixed after a reboot of the secondary node, while troubleshooting another issue. Difficult to understand what happened

Thanks anyway!

[anettekr]

Hello,  i know this is an old post but hoping you might be able to help.

i am having some issues pin pointing connection resets for a payment application.  trying to find out if the firewall is at fault so something else is along the path.

i'm seeing First Packet isn't SYN being dropped by the firewall with the flag -FIN-PUSH-ACK.  the timings of these drops don't match exactly with the connection errors seen for the payment application but there are some of these drops around the same time as connection problems.  we also see them regularly throughout the day.

not done any work to resolve just yet as not sure what the best approach would be seen various 'solutions': change TCP end Timeout, exclude IPs from SecureXL.   disable HTTPsi, its enabled but connection is already excluded as its payment related.  Failover firewalls.

would you have any suggestions based on the flag we see, FIN-PUSH-ACK

Many thanks,

Anette

[Timothy_Hall]

Unlikely that the FIN-PUSH-ACK drops are related to your suspected RST issue, as the presence of those flags reveals that the host already believes the connection is over and is trying to end it gracefully.  That connection no longer being present in the firewall's state table and the drop occurring shouldn't really affect that.  I suppose the payment app may hang waiting for the connection to gracefully end before it is able to launch a new connection and continue normal operation, and that may look like an interruption/outage.

It is also possible that the long-running payment application connection is getting silently timed out by the firewall, the application realizes this and tries to close it gracefully but gets stuck.  If you suspect this is the case you could try increasing the timeout for the relevant TCP service it is using, or try changing fw_rst_expired_conn from 0 to 1.  This will cause a RST to be sent immediately upon expiration, which may help the host figure out the connection is gone and to launch a new one immediately.

[anettekr]

Thank you for the reply, much appreciated ill do some digging into what you've mentioned. 

i am reluctant to alter the service as its HTTPs and not sure if it would end up impacting other connections. and will see what the impact could be for changing the  fw_rst_expired_conn setting.