topic Re: tcp rst management in Firewall and Security Management

tcp rst management

Luis_Miguel_Mig — Fri, 28 Jun 2024 17:17:20 GMT

We are seeing a case where a firewall receives a tcp reset from the client host and keep it(doesn't send it to the server) and doesn't remove the tcp session from the tcp session table. Why would do it?

This is case of a load balancer configured to do tcp half close check so it send tcp resets after receiving a sin ack from the server.

However the firewall keep the RESET and doesn't close the session in the session table and therefore the loadbalancer keeps sending RESETS and the server keeps sending SYN-ACKs ....

fwaccel conns shows me all the connections in SYNC-ACK received state

Note: It happens all the time. This behavior is not related with Policy Installation and connection persistency

Re: tcp rst management

PhoneBoy — Fri, 28 Jun 2024 19:14:28 GMT

TCP connections are generally hung onto for 25 seconds after closing.
This is by design.
https://support.checkpoint.com/results/sk/sk110672

Re: tcp rst management

Lesley — Fri, 28 Jun 2024 19:35:28 GMT

TCP connections

If the TCP connection doesn't complete the 3-way handshake within the TCP Start Timeout (25 seconds, by default).
After the TCP End Timeout (20 seconds, by default), which applies after receiving two FIN packets (one in each direction: client-to-server, and server-to-client) or an RST packet. This allows stray ACK packets that belong to the connection, but may arrive late.
Note: In R80.20, the TCP End Timeout (for R80.20 gateway and above) was changed to 5 sec from 20 sec. The End (connection) Timeout describes the connection expiration once the connection reaches both FIN (received FIN from both sides or RST) – aka, terminated. This timeout allow us to handle retransmissions post connection termination. We found that most of the connections found in the connection table are terminated connections that wait to be expired. In order to improve gateway performance (reduce memory consumption and improve table lookup), we reduced the timeout to 5 sec (from 20).
If the connection is idle (no packets received) for the TCP Session Timeout (3600 seconds, by default)
If Aggressive Aging is enabled in the IPS Blade, the Aggressive Aging timeouts will apply if the connection table is near capacity.
If the connection is dropped due a policy violation with one of the Software Blades (e.g. App Control rule or IPS).

UDP sessions

If the session is idle (no packets received) for the UDP Virtual Session Timeout (40 seconds, by default).
If the connection is dropped due a policy violation with one of the Software Blades (e.g. App Control rule or IPS).

ICMP sessions

If the session is idle (no packets received) for the ICMP Virtual Session Timeout (30 seconds, by default).
If the connection is dropped due a policy violation with one of the Software Blades (e.g. App Control rule or IPS).

TCP, UDP, and ICMP session timers can be configured in 'Global Properties > Stateful Inspection'.

https://support.checkpoint.com/results/sk/sk41248

Re: tcp rst management

the_rock — Fri, 28 Jun 2024 19:45:25 GMT

I believe what both @Lesley and @PhoneBoy provided is valid in your case.

Andy

Re: tcp rst management

Luis_Miguel_Mig — Mon, 01 Jul 2024 11:46:22 GMT

It looks like the checkpoint keeps the tcp session in the session table for about 30 seconds and it consistently drops the TCP RESET from the client. Perhaps because the 3-way handshake never gets established.
The client is configured to do a tcp half open helth check so it always sends a tcp reset after syn, syn ack.

Those 30 seconds are close to the TCP start time out (25)+ TPC End Timeout (5)

tcpdump on the client side

11:24:32.453451 IP client.ip > server_ip.server_port: Flags [S], seq 2125582756, win 512, length 0
11:24:32.453617 IP server_ip.server_port > client.ip: Flags [S.], seq 3760118362, ack 2125582757, win 29200, options [mss 1460], length 0
11:24:33.953414 IP server_ip.server_port > client.ip: Flags [S.], seq 3760118362, ack 2125582757, win 29200, options [mss 1460], length 0
11:24:35.953369 IP server_ip.server_port > client.ip: Flags [S.], seq 3760118362, ack 2125582757, win 29200, options [mss 1460], length 0
11:24:39.953411 IP server_ip.server_port > client.ip: Flags [S.], seq 3760118362, ack 2125582757, win 29200, options [mss 1460], length 0
11:24:47.953370 IP server_ip.server_port > client.ip: Flags [S.], seq 3760118362, ack 2125582757, win 29200, options [mss 1460], length 0
11:25:03.953390 IP server_ip.server_port > client.ip: Flags [S.], seq 3760118362, ack 2125582757, win 29200, options [mss 1460], length 0

tcpdump on the external server side

11:24:32.453312 IP client.ip > server_ip.server_port: Flags [S], seq 2125582756, win 512, length 0
11:24:32.453627 IP server_ip.server_port > client.ip: Flags [S.], seq 3760118362, ack 2125582757, win 29200, options [mss 1460], length 0
11:24:32.453962 IP client.ip > server_ip.server_port: Flags [R], seq 2125582757:2125582817, win 0, length 60
11:24:33.953426 IP server_ip.server_port > client.ip: Flags [S.], seq 3760118362, ack 2125582757, win 29200, options [mss 1460], length 0
11:24:33.956704 IP client.ip > server_ip.server_port: Flags [R.], seq 1:47, ack 1, win 0, length 46
11:24:35.953383 IP server_ip.server_port > client.ip: Flags [S.], seq 3760118362, ack 2125582757, win 29200, options [mss 1460], length 0
11:24:35.954858 IP client.ip > server_ip.server_port: Flags [R.], seq 1:47, ack 1, win 0, length 46
11:24:39.953426 IP server_ip.server_port > client.ip: Flags [S.], seq 3760118362, ack 2125582757, win 29200, options [mss 1460], length 0
11:24:39.955255 IP client.ip > server_ip.server_port: Flags [R.], seq 1:47, ack 1, win 0, length 46
11:24:47.953382 IP server_ip.server_port > client.ip: Flags [S.], seq 3760118362, ack 2125582757, win 29200, options [mss 1460], length 0
11:24:47.953631 IP client.ip > server_ip.server_port: Flags [R.], seq 1:47, ack 1, win 0, length 46
11:25:03.953402 IP server_ip.server_port > client.ip: Flags [S.], seq 3760118362, ack 2125582757, win 29200, options [mss 1460], length 0
11:25:03.953505 IP client.ip > server_ip.server_port: Flags [R.], seq 1:47, ack 1, win 0, length 46

Re: tcp rst management

the_rock — Mon, 01 Jul 2024 12:14:33 GMT

Its 100% because 3 way handshake is failing, but we need to be sure which end is the one resetting the connection. And now, I will go celebrate Canada day 🙂

Cheers mate.

Andy

Re: tcp rst management

Luis_Miguel_Mig — Mon, 01 Jul 2024 15:26:55 GMT

The client (loadbalancer) sends the reset by design. It is the typical tcp half open health check sent by load balancers.

Re: tcp rst management

Luis_Miguel_Mig — Tue, 02 Jul 2024 09:09:30 GMT

In other words, checkpoint default configuration doesn't interoperate well will typical loadbalancers tcp half open checks