If you check the log you might find that your connections to the inactive cluster member are getting dropped as out-of-state or spoofed. Likely, your connection to the inactive member and the return traffic from it are on different interfaces on the inactive member. Traffic _to_ the inactive member wants to go via the active member.
How I fixed this in my environment was to put static routes on the internal router that is adjacent to the cluster.
Suppose your mgmt segment is 10.2.65.0/24 and those are the addresses you are connecting to with the web GUI.
And, 10.2.44.0/24 is the backbone segment between the gateways and your adjacent internal router.
Let's say 10.2.65.1 and 2 are the mgmt IPs of the two gateways, and 10.2.44.5 and 6 are the IPs of (say) eth1 of the two gateways.
Router has:
ip route 10.2.65.1 255.255.255.255 10.2.44.5
ip route 10.2.65.2 255.255.255.255 10.2.44.6
If you don't do this, this adjacent router sends all of the traffic for dest 10.2.44.0/24 to the active member. The traffic might still reach the inactive member after coming out the other side of the active member, but the reply from the inactive member will come out a different interface on it, hence will be dropped.