Finally, I was able to identify the root cause.
It seems the firewall is dealing with a concurrency issue and for some traffic flows the firewall sends an RST packet to clear the inconsistent connection.
The issue happened in this flow: Client 172.20.4.2(TCP/57490) -> Server 172.16.10.36(TCP/135), both the client and server were using Windows LNB in unicast mode.
While the fw was still processing the first SYN packet, simultaneously received an SYN+ACK for the same connection, because the processing of the first SYN wasn’t finished the SYN+ACK was discarded. This behavior is visible In the kernel fw debug and in the fw monitor.
After discarding the SYN+ACK packet the fw kept trying to process the following packets of the connection, after receiving an PUSH+ACK and because it was still waiting for the ACK to the first SYN, it sends a RST packet to both parties to clear the connection. Because this inspection is being done in L2 mode, the mac address used by the firewall is the MAC address of the client (Unicast NBL mac) thus allowing the switch to wrongly learn the mac address of the NLB cluster in the switch port connected to the firewall.
SYN-ACK discard
[-- Stateful VM inbound: Entering (1689918324) --];
@;27479832;21Jul2023 6:45:24.620991;[cpu_24];[fw4_14];fwconn_key_set_links_inbound: create link cls_o <dir 1, 172.16.10.36:135 -> 172.20.4.2:57490 IPP 6> -> <dir 0, 172.20.4.2:57490 -> 172.16.10.36:135 IPP 6>(0x5);
@;27479832;21Jul2023 6:45:24.620996;[cpu_4];[fw4_23];Before VM: <dir 0, 172.16.10.36:135 -> 172.20.4.2:57490 IPP 6> (len=52) TCP flags=0x12 (SYN-ACK), seq=2770938187, ack=496573280, data end=2770938188 (ifn=19) (first seen) ;
@;27479832;21Jul2023 6:45:24.620999;[cpu_24];[fw4_14];fwconn_key_set_link (normal): conn <dir 1, 172.16.10.36:135 -> 172.20.4.2:57490 IPP 6> added to dispatcher with ppack_id=-1 (qid=-1), member_id=-1, flags=1;
@;27479832;21Jul2023 6:45:24.621001;[cpu_4];[fw4_23];fwconn_lookup_cache: conn <dir 0, 172.16.10.36:135 -> 172.20.4.2:57490 IPP 6>;
@;27479832;21Jul2023 6:45:24.621004;[cpu_4];[fw4_23];fwconn_lookup_cache: not found in connections table. nvals = 0;
@;27479832;21Jul2023 6:45:24.621009;[cpu_24];[fw4_14];fwconn_key_set_links_inbound: create link srs_o <dir 1, 172.20.4.2:57490 -> 172.16.10.36:135 IPP 6> -> <dir 0, 172.20.4.2:57490 -> 172.16.10.36:135 IPP 6>(0x2);
@;27479832;21Jul2023 6:45:24.621011;[cpu_4];[fw4_23];fw_first_packet_state_checks: Executing accept_non_syn inspect scripts (internal id=216, user id=-1);
@;27479832;21Jul2023 6:45:24.621015;[cpu_4];[fw4_23];fwconnoxid_msg_get_cliconn: No connoxid message on <dir 1, 172.16.10.36:135 -> 172.20.4.2:57490 IPP 6>;
@;27479832;21Jul2023 6:45:24.621016;[cpu_24];[fw4_14];fwconn_key_set_link (normal): conn <dir 1, 172.20.4.2:57490 -> 172.16.10.36:135 IPP 6> added to dispatcher with ppack_id=-1 (qid=-1), member_id=-1, flags=1;
@;27479832;21Jul2023 6:45:24.621018;[cpu_24];[fw4_14];fwconn_key_set_links_inbound: VM-side links created successfuly;
@;27479832;21Jul2023 6:45:24.621020;[cpu_24];[fw4_14];fwconn_key_init_links: connection completed, unset PARTIAL_SETUP bit;
@;27479832;21Jul2023 6:45:24.621024;[cpu_24];[fw4_14];fwconnoxid_msg_get_cliconn: No connoxid message on <dir 1, 172.20.4.2:57490 -> 172.16.10.36:135 IPP 6>;
@;27479832;21Jul2023 6:45:24.621030;[cpu_4];[fw4_23];fwconn_lookup_other_ex__legacy__: conn <dir 0, 172.20.4.2:57490 -> 172.16.10.36:135 IPP 6>
not found in connections table;
@;27479832;21Jul2023 6:45:24.621033;[cpu_24];[fw4_14];fwconn_key_init_links: Creating links (outbound). One way links=0, Replies from any=0;
@;27479832;21Jul2023 6:45:24.621035;[cpu_4];[fw4_23];fwconn_key_lookup_ex: conn <dir 0, 172.20.4.2:57490 -> 172.16.10.36:135 IPP 6> not found in connections table.;
@;27479832;21Jul2023 6:45:24.621038;[cpu_24];[fw4_14];fwconn_key_set_links_outbound: create link srs_i <dir 0, 172.16.10.36:135 -> 172.20.4.2:57490 IPP 6> -> <dir 0, 172.20.4.2:57490 -> 172.16.10.36:135 IPP 6>(0x6);
@;27479832;21Jul2023 6:45:24.621040;[cpu_4];[fw4_23];FW-1: fw_log_tcp_out_of_state: reason First packet isn't SYN. th_flags 0x12;
@;27479832;21Jul2023 6:45:24.621043;[cpu_4];[fw4_23];fw_first_packet_state_checks: Dropping out of state TCP packet;
@;27479832;21Jul2023 6:45:24.621046;[cpu_24];[fw4_14];fwconn_key_set_link (normal): conn <dir 0, 172.16.10.36:135 -> 172.20.4.2:57490 IPP 6> added to dispatcher with ppack_id=-1 (qid=-1), member_id=-1, flags=1;
@;27479832;21Jul2023 6:45:24.621046;[cpu_4];[fw4_23];fw_first_packet_state_checks: chain->dir=0, dir=0, th_flags=0x12, src=172.16.10.36, dst=172.20.4.2;
@;27479832;21Jul2023 6:45:24.621051;[cpu_24];[fw4_14];fwconn_key_set_links_outbound: server-side links created successfuly;
@;27479832;21Jul2023 6:45:24.621052;[cpu_24];[fw4_14];fw_first_packet_outbound_init: Setting DONE_OUTBOUND_INIT flag;
@;27479832;21Jul2023 6:45:24.621053;[cpu_4];[fw4_23];fw_log_drop_ex: Packet proto=6 172.16.10.36:135 -> 172.20.4.2:57490 dropped by fw_first_packet_state_checks Reason: First packet isn't SYN;
@;27479832;21Jul2023 6:45:24.621056;[cpu_4];[fw4_23];fw_handle_first_packet: first packet state violation (action=DROP);
@;27479832;21Jul2023 6:45:24.621061;[cpu_4];[fw4_23];After VM: <dir 0, 172.16.10.36:135 -> 172.20.4.2:57490 IPP 6> (len=52) TCP flags=0x12 (SYN-ACK), seq=2770938187, ack=496573280, data end=2770938188 ;
@;27479832;21Jul2023 6:45:24.621061;[cpu_24];[fw4_14];After VM: <dir 0, 172.20.4.2:57490 -> 172.16.10.36:135 IPP 6> (len=52) TCP flags=0x2 (SYN), seq=496573279, ack=0, data end=496573280 ;
@;27479832;21Jul2023 6:45:24.621064;[cpu_24];[fw4_14];VM Final action=ACCEPT;
@;27479832;21Jul2023 6:45:24.621065;[cpu_4];[fw4_23];VM Final action=DROP;
@;27479832;21Jul2023 6:45:24.621066;[cpu_4];[fw4_23]; ----- Stateful VM inbound Completed -----
Connection reset:
@;27479832;21Jul2023 6:45:24.621942;[cpu_24];[fw4_14];
[-- Stateful VM inbound: Entering (1689918324) --];
@;27479832;21Jul2023 6:45:24.621945;[cpu_24];[fw4_14];Before VM: <dir 0, 172.16.10.36:135 -> 172.20.4.2:57490 IPP 6> (len=148) TCP flags=0x18 (PUSH-ACK), seq=2770938188, ack=496573440, data end=2770938296 (ifn=19) (first seen) ;
@;27479832;21Jul2023 6:45:24.621947;[cpu_24];[fw4_14];fwconn_lookup_cache: conn <dir 0, 172.16.10.36:135 -> 172.20.4.2:57490 IPP 6>;
@;27479832;21Jul2023 6:45:24.621954;[cpu_24];[fw4_14];<10001,40046080,1,e10,64,UUID: 64ba1b74-0001-0001-bd-90-8d-ee-fa-dd-ec-d8, 13,ffffffff,ffffffff,11,2007800,7c800,80000084,OPQS:[ffffc900b9e11648,ffffc900b950fc90,0,ffffc900b868bab0,0,0,ffffc900c0e7dad8,0,0,0,ffffc900bad1be88,0,0,0,0,0,ffffc900bc39aee0,0,0,0],0,0,0,0,0,0,0,0,0,0,0,24855800,0,0>
found in connections table dir=2 keytype=806;
@;27479832;21Jul2023 6:45:24.621956;[cpu_24];[fw4_14];fw_tcp_state_verification: Received ACK as response to SYN packet. Sending RST on behalf of the client - tcp flags=18, ctype=10001, cflags=40046080;
@;27479832;21Jul2023 6:45:24.621959;[cpu_24];[fw4_14];fw_log_drop_ex: Packet proto=6 172.16.10.36:135 -> 172.20.4.2:57490 dropped by fw_tcp_state_verification Reason: ACK response to SYN. Sending RST on behalf of the client;
@;27479832;21Jul2023 6:45:24.621960;[cpu_24];[fw4_14];fw_conn_inspect: fw_tcp_state_verification returned REJECT;
@;27479832;21Jul2023 6:45:24.621962;[cpu_24];[fw4_14];After VM: <dir 0, 172.16.10.36:135 -> 172.20.4.2:57490 IPP 6> (len=148) TCP flags=0x18 (PUSH-ACK), seq=2770938188, ack=496573440, data end=2770938296 ;
@;27479832;21Jul2023 6:45:24.621963;[cpu_24];[fw4_14];VM Final action=REJECT;
@;27479832;21Jul2023 6:45:24.621964;[cpu_24];[fw4_14]; ----- Stateful VM inbound Completed -----
FW monitor
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.620576 bond1:i10 (Stateless verifications (in))[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.620587 bond1:i11 (fw multik misc proto forwarding)[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.620599 bond1:i12 (fw VM inbound )[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_23] 21Jul2023 6:45:24.620950 bond1:i9 (IP Options Strip (in))[44]: 172.16.10.36 -> 172.20.4.2 (TCP) len=52 id=11729
TCP: 135 -> 57490 .S..A. seq=a529294b ack=1d991b60
[vs_0][fw_23] 21Jul2023 6:45:24.620963 bond1:i10 (Stateless verifications (in))[44]: 172.16.10.36 -> 172.20.4.2 (TCP) len=52 id=11729
TCP: 135 -> 57490 .S..A. seq=a529294b ack=1d991b60
[vs_0][fw_23] 21Jul2023 6:45:24.620976 bond1:i11 (fw multik misc proto forwarding)[44]: 172.16.10.36 -> 172.20.4.2 (TCP) len=52 id=11729
TCP: 135 -> 57490 .S..A. seq=a529294b ack=1d991b60
[vs_0][fw_23] 21Jul2023 6:45:24.620989 bond1:i12 (fw VM inbound )[44]: 172.16.10.36 -> 172.20.4.2 (TCP) len=52 id=11729
TCP: 135 -> 57490 .S..A. seq=a529294b ack=1d991b60
[vs_0][fw_14] 21Jul2023 6:45:24.621084 bond1:I13 (fw SCV inbound)[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.621097 bond1:I14 (fw offload inbound)[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.621125 bond1:I15 (fw post VM inbound )[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.621170 bond1:I16 (passive streaming (in))[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.621189 bond1:I17 (TCP streaming (in))[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.621201 bond1:I18 (IP Options Restore (in))[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.621211 bond1:I19 (Chain End)[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.621243 eth3-03:o0 (IP Options Strip (out))[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.621262 eth3-03:o1 (TCP streaming (out))[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.621281 eth3-03:o2 (passive streaming (out))[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.621290 eth3-03:o3 (Stateless verifications (out))[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.621300 eth3-03:o4 (fw VM outbound)[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.621323 eth3-03:O5 (fw post VM outbound )[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.621345 eth3-03:O6 (TCP streaming post VM)[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.621355 eth3-03:O7 (IP Options Restore (out))[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.621365 eth3-03:O13 (Chain End)[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=52 id=10946
TCP: 57490 -> 135 .S.... seq=1d991b5f ack=00000000
[vs_0][fw_14] 21Jul2023 6:45:24.621407 bond1:i9 (IP Options Strip (in))[40]: 172.20.4.2 -> 172.16.10.36 (TCP) len=40 id=10947
TCP: 57490 -> 135 ....A. seq=1d991b60 ack=a529294c
[vs_0][fw_14] 21Jul2023 6:45:24.621439 bond1:i10 (Stateless verifications (in))[40]: 172.20.4.2 -> 172.16.10.36 (TCP) len=40 id=10947
TCP: 57490 -> 135 ....A. seq=1d991b60 ack=a529294c
[vs_0][fw_14] 21Jul2023 6:45:24.621462 bond1:i11 (fw multik misc proto forwarding)[40]: 172.20.4.2 -> 172.16.10.36 (TCP) len=40 id=10947
TCP: 57490 -> 135 ....A. seq=1d991b60 ack=a529294c
[vs_0][fw_14] 21Jul2023 6:45:24.621481 bond1:i12 (fw VM inbound )[40]: 172.20.4.2 -> 172.16.10.36 (TCP) len=40 id=10947
TCP: 57490 -> 135 ....A. seq=1d991b60 ack=a529294c
[vs_0][fw_14] 21Jul2023 6:45:24.621510 bond1:I13 (fw SCV inbound)[40]: 172.20.4.2 -> 172.16.10.36 (TCP) len=40 id=10947
TCP: 57490 -> 135 ....A. seq=1d991b60 ack=a529294c
[vs_0][fw_14] 21Jul2023 6:45:24.621520 bond1:I14 (fw offload inbound)[40]: 172.20.4.2 -> 172.16.10.36 (TCP) len=40 id=10947
TCP: 57490 -> 135 ....A. seq=1d991b60 ack=a529294c
[vs_0][fw_14] 21Jul2023 6:45:24.621530 bond1:I15 (fw post VM inbound )[40]: 172.20.4.2 -> 172.16.10.36 (TCP) len=40 id=10947
TCP: 57490 -> 135 ....A. seq=1d991b60 ack=a529294c
[vs_0][fw_14] 21Jul2023 6:45:24.621606 bond1:i9 (IP Options Strip (in))[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=200 id=10948
TCP: 57490 -> 135 ...PA. seq=1d991b60 ack=a529294c
[vs_0][fw_14] 21Jul2023 6:45:24.621631 bond1:i10 (Stateless verifications (in))[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=200 id=10948
TCP: 57490 -> 135 ...PA. seq=1d991b60 ack=a529294c
[vs_0][fw_14] 21Jul2023 6:45:24.621651 bond1:i11 (fw multik misc proto forwarding)[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=200 id=10948
TCP: 57490 -> 135 ...PA. seq=1d991b60 ack=a529294c
[vs_0][fw_14] 21Jul2023 6:45:24.621670 bond1:i12 (fw VM inbound )[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=200 id=10948
TCP: 57490 -> 135 ...PA. seq=1d991b60 ack=a529294c
[vs_0][fw_14] 21Jul2023 6:45:24.621697 bond1:I13 (fw SCV inbound)[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=200 id=10948
TCP: 57490 -> 135 ...PA. seq=1d991b60 ack=a529294c
[vs_0][fw_14] 21Jul2023 6:45:24.621707 bond1:I14 (fw offload inbound)[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=200 id=10948
TCP: 57490 -> 135 ...PA. seq=1d991b60 ack=a529294c
[vs_0][fw_14] 21Jul2023 6:45:24.621717 bond1:I15 (fw post VM inbound )[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=200 id=10948
TCP: 57490 -> 135 ...PA. seq=1d991b60 ack=a529294c
[vs_0][fw_14] 21Jul2023 6:45:24.621884 bond1:i9 (IP Options Strip (in))[44]: 172.16.10.36 -> 172.20.4.2 (TCP) len=148 id=11730
TCP: 135 -> 57490 ...PA. seq=a529294c ack=1d991c00
[vs_0][fw_14] 21Jul2023 6:45:24.621903 bond1:i10 (Stateless verifications (in))[44]: 172.16.10.36 -> 172.20.4.2 (TCP) len=148 id=11730
TCP: 135 -> 57490 ...PA. seq=a529294c ack=1d991c00
[vs_0][fw_14] 21Jul2023 6:45:24.621922 bond1:i11 (fw multik misc proto forwarding)[44]: 172.16.10.36 -> 172.20.4.2 (TCP) len=148 id=11730
TCP: 135 -> 57490 ...PA. seq=a529294c ack=1d991c00
[vs_0][fw_14] 21Jul2023 6:45:24.621941 bond1:i12 (fw VM inbound )[44]: 172.16.10.36 -> 172.20.4.2 (TCP) len=148 id=11730
TCP: 135 -> 57490 ...PA. seq=a529294c ack=1d991c00
[vs_0][fw_14] 21Jul2023 6:45:24.622144 bond1:i9 (IP Options Strip (in))[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=208 id=10949
TCP: 57490 -> 135 ...PA. seq=1d991c00 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622173 bond1:i10 (Stateless verifications (in))[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=208 id=10949
TCP: 57490 -> 135 ...PA. seq=1d991c00 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622198 bond1:i11 (fw multik misc proto forwarding)[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=208 id=10949
TCP: 57490 -> 135 ...PA. seq=1d991c00 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622218 bond1:i12 (fw VM inbound )[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=208 id=10949
TCP: 57490 -> 135 ...PA. seq=1d991c00 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622246 bond1:I13 (fw SCV inbound)[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=208 id=10949
TCP: 57490 -> 135 ...PA. seq=1d991c00 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622256 bond1:I14 (fw offload inbound)[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=208 id=10949
TCP: 57490 -> 135 ...PA. seq=1d991c00 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622265 bond1:I15 (fw post VM inbound )[44]: 172.20.4.2 -> 172.16.10.36 (TCP) len=208 id=10949
TCP: 57490 -> 135 ...PA. seq=1d991c00 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622326 bond1:i9 (IP Options Strip (in))[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622345 bond1:i10 (Stateless verifications (in))[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622363 bond1:i11 (fw multik misc proto forwarding)[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622381 bond1:i12 (fw VM inbound )[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622412 bond1:I13 (fw SCV inbound)[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622421 bond1:I14 (fw offload inbound)[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622431 bond1:I15 (fw post VM inbound )[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622462 bond1:I16 (passive streaming (in))[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622473 bond1:I17 (TCP streaming (in))[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622483 bond1:I18 (IP Options Restore (in))[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622493 bond1:I19 (Chain End)[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622517 eth3-03:o0 (IP Options Strip (out))[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622535 eth3-03:o1 (TCP streaming (out))[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622553 eth3-03:o2 (passive streaming (out))[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622564 eth3-03:o3 (Stateless verifications (out))[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622574 eth3-03:o4 (fw VM outbound)[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622593 eth3-03:O5 (fw post VM outbound )[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622614 eth3-03:O6 (TCP streaming post VM)[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622624 eth3-03:O7 (IP Options Restore (out))[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490..R... seq=a52929b8 ack=a52929b8
[vs_0][fw_14] 21Jul2023 6:45:24.622633 eth3-03:O13 (Chain End)[40]: 172.16.10.36 -> 172.20.4.2 (TCP) len=40 id=11731
TCP: 135 -> 57490 ..R... seq=a52929b8 ack=a52929b8
The correct fix for this issue should be to configure the NLB in multicast igmp mode instead of unicast, as an emergency measure one could try to disable the stateful inspection associated with these servers as mentioned in this topic.