Let me guess, if you use HTTPS/TLS as the VPN transport instead of IPSec, performance is just great.
You have a low MTU in your network path somewhere, or somehow the VPN client is affecting the MTU when it is active. The symptom of this is terrible performance due to packet loss because of the inability to fragment IPSec traffic due to the DF bit being set.
To verify, run netstat -sv in Windows and note the counters associated with IP frags and TCP segment retransmissions. Initialize the VPN tunnel with IPSec and start a big TCP-based upload. Which frag/retransmit counters in the netstat -sv output jump? This should give you some idea of where to look.
Attend my online "Be your Own TAC: Part Deux" CheckMates event
March 27th with sessions for both the EMEA and Americas time zones