Unable to connect to remote server through site to site vpn

[gouri-menon]

Hi team. I am trying to setup a site-to-site VPN across 2 tenants in Azure. The product I am using in both location is "CloudGuard Network Security Firewall & Threat Prevention" from Azure marketplace. I did setup the VPN in mesh topology with local & remote "interoperable" device along with the shared key. I am doing this to simulate not having access to the remote checkpoint. I then setup the policy rule & NAT rule to flow over the VPN community.

I then tried connecting from a windows server on side to a NGINX server on the other side of the tunnel 

  -  172.16.102.213 (WIN) ==> GW01 ==> VPN Tunnel ==> GW02 ==> 10.0.0.100 (NGINX)

In the logs (using smartview) I can see the session which uses VPN blade + encrypt on the source gateway & VPN blade + decrypt on the destination gateway along with the service (http or https). I have even tried the other way for port 3389 & here too I can see it go through the tunnel & arrive at the remote site. In both cases however, the packet is not flowing from the gateway to the destination machine. 

# vpn tu

Option-1

Peer xxx.xxx.xxx.xxx , gw-01 SAs:

IKE SA <0aa90a313c5066a6,69958fa937977e7d>

Option-2

SAs of all instances:

Peer xxx.xxx.xxx.xxx , gw-01 SAs:

IKE SA <0aa90a313c5066a6,69958fa937977e7d>
(No IPSec SAs)

Similarly, it shows the tunnel up on the other gateway. Am I missing a step or doing something wrong ? Any help would be greatly appreciated.

[Lesley]

If you think the problem is not the vpn tunnel I would do ip r get IP and use that interface output to tcpdump to see what happens to the traffic. Could be routing or acl on system itself. 

[gouri-menon]

Thank you for your response. I did run the capture on destination firewall. I can see the SYN packet coming from the remote site. However, I do not see that packet being sent to the actual server. In the logs, I do not see a drop either. That is what is puzzling. I checked connection from the firewall to backend server using telnet <IP> <port> and that works fine.
In the interest of time, I am checking if I can get the route-based VPN (DMVPN) working. Will post here once I test.