At that time of night there was not a lot of traffic that was being passed through the firewall. I had waited over an hour trying to troubleshoot and restarted my own connection attempts several times to no avail
rule 41 is simply
src: internal networks dst: 192.168.209.0/24 service:any action:allow
This particular piece of traffic does not use an automatic NAT currently there is no specific rule that has been created for this traffic to pass after it has been NAT'd.
However, this traffic passes after NAT without issues and does not need a rule on the current firewall.
I have searched the policy for 192.168.209.1 in the source field permitting the destination network of 192.168.209.17
here is a capture that i had gotten from the working FW
bond20.6:i0 (IP Options Strip (in))[1500]:10.30.185.17 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond20.6:i1 (Stateless verifications (in))[1500]:10.30.185.17 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond20.6:i2 (fw multik misc proto forwarding)[1500]:10.30.185.17 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond20.6:i3 (SecureXL conn sync)[1500]:10.30.185.17 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond20.6:i4 (fw VM inbound )[1500]:10.30.185.17 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond20.6:I5 (SecureXL inbound)[1500]:10.30.185.17 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond20.6:I6 (fw SCV inbound)[1500]:10.30.185.17 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond20.6:I7 (passive streaming (in))[1500]:10.30.185.17 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond20.6:I8 (TCP streaming (in))[1500]:10.30.185.17 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond20.6:I9 (IP Options Restore (in))[1500]:10.30.185.17 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond20.6:I10 (Chain End)[1500]:10.30.185.17 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond10.609:o0 (IP Options Strip (out))[1500]:10.30.185.17 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond10.609:o1 (TCP streaming (out))[1500]:10.30.185.17 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond10.609:o2 (passive streaming (out))[1500]:10.30.185.17 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond10.609:o3 (Stateless verifications (out))[1500]:10.30.185.17 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond10.609:o4 (fw VM outbound)[1500]:10.30.185.17 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond10.609:O5 (SecureXL outbound)[1500]:192.168.209.1 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond10.609:O6 (TCP streaming post VM)[1500]:192.168.209.1 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond10.609:O7 (IP Options Restore (out))[1500]:192.168.209.1 -> 192.168.209.17 (TCP) len=1500 id=24939;
bond10.609:O8 (Chain End)[1500]:192.168.209.1 -> 192.168.209.17 (TCP) len=1500 id=24939;
As you can see it appears to process correctly and passes the traffic on without issues and i get the web page to return.
However if you look at the capture i have from the non working fw below
bond20.6:i0 (IP Options Strip (in))[52]:10.30.185.17 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:i1 (Stateless verifications (in))[52]:10.30.185.17 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:i2 (fw multik misc proto forwarding)[52]:10.30.185.17 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:i3 (SecureXL conn sync)[52]:10.30.185.17 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:i4 (fw VM inbound )[52]:10.30.185.17 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:I5 (SecureXL inbound)[52]:10.30.185.17 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:I6 (fw SCV inbound)[52]:10.30.185.17 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:I7 (passive streaming (in))[52]:10.30.185.17 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:I8 (TCP streaming (in))[52]:10.30.185.17 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:I9 (IP Options Restore (in))[52]:10.30.185.17 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:I10 (Chain End)[52]:10.30.185.17 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:o0 (IP Options Strip (out))[52]:10.30.185.17 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:o1 (TCP streaming (out))[52]:10.30.185.17 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:o2 (passive streaming (out))[52]:10.30.185.17 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:o3 (Stateless verifications (out))[52]:10.30.185.17 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:o4 (fw VM outbound)[52]:10.30.185.17 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:O5 (SecureXL outbound)[52]:192.168.209.1 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:O6 (TCP streaming post VM)[52]:192.168.209.1 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:O7 (IP Options Restore (out))[52]:192.168.209.1 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:O8 (Chain End)[52]:192.168.209.1 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:i0 (IP Options Strip (in))[52]:192.168.209.1 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:i1 (Stateless verifications (in))[52]:192.168.209.1 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:i2 (fw multik misc proto forwarding)[52]:192.168.209.1 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:i3 (SecureXL conn sync)[52]:192.168.209.1 -> 192.168.209.17 (TCP) len=52 id=26209;
bond20.6:i4 (fw VM inbound )[52]:192.168.209.1 -> 192.168.209.17 (TCP) len=52 id=26209;
It appears to successfully go through the firewall in the first section of the capture then immediately after i see it looks like it is processed against the chain again and ultimately dropped against the FW VM Inbound which i presume is the no rule drop.
If it does require a specific rule due to being a manual NAT then i dont understand how the current firewall operates and passes the traffic.
open to further comments and being told im wrong, at least then i can fix it.
TIA