Check your logs, I bet you'll see lots of out-of-state drops. If you do, the issue is asymmetric routing. Your C2S packets go from the client to their local gateway and over to the server via the Cisco devices, then the S2C packet goes to the server local gateway that never saw the C2S packet, hence doesn't have the connection in its tables and is dropping it.
If that's the case, I suggest that the IPVPN be moved to dedicated interfaces/subnets rather than sharing the client subnet, so all packets must traverse both gateways in both directions.