Good morning everyone.
We have a situation within a customers environment where several TCP-connections between two communication-partners from one SrcIP to one DestIP:DestPort are passed over two Checkpoint Firewalls - hereinafter referred to as "A" and "B".
When the SrcIP sends a SYN-packet to open a new connection, some of these new connection attempts are getting dropped silently by the Firewall "A".
According to our past analysis - and as of my understandigs - the reason is as follows:
The first Firewall "A" is silently-dropping the SYN-packet because an entry to an already established connection (five-tuple) already exists and therefore matches in its connection table, while it does not receive a response on its initially started "Smart Connection Reuse" Feature.
As I understood so far: the Firewall "A" converts the SYN-packet into an ACK-packet in that process which is forwarded to the DestIP:DestPort to check if the already existing connection is still in use or not by awaiting either an ACK or a RST as a response.
Now as of the setup of our infrastructure: after the ACK-packet is leaving Firewall "A" it hits Firewall "B" first, before reaching the DestIP that is located behind that.
On Firewall "B" we could see that the ACK-packet passed through both of its (incoming and outgoing) interfaces. [As of my unterstanding, this indicates that an entry in Firewall "B's" connection table for the already existing five-tuple-connection must match there as well. Otherwise the ACK-packet would have been dropped with a log-message saying "first packet not SYN" ...is that correct??...]
Further analysis showed that the DestIP answers with a RST-Packet back to Firewall "B", but Firewall "B" does not forward that RST-packet back to Firewall "A". Within these RST-packets we have noticed that mostly all of its "TCP-Parameters" - for example its Sequencenumber - equal NULL. [Could that be an issue leading Firewall "B" to not returning this RST-packet back to "A"? Like e.g.: "it is not able to sort/order this TCP-packet as response to the before passed ACK?"]
Meanwhile, the SrcIP does retransmit its former SYN-Packet twice, which just repeats the above illustrated process, and stops trying it after 7000ms passed by according to its local settings in the TCP-stack.
So right now we are focusing on the question:
- How does the Smart Connection Reuse Feature exactly behave, when it applies for one (five-tuple) connection that is routed and passing over two checkpoint firewalls?
...and additional:
- Is there already a solution for such scenarios that I am currently not aware of?
Thanks in advance for any further questions, answers and advices.