After the user restarted the issue came back.
I had to use Safe Mode to remove it. But you cannot just run MSI files. While in Safe Mode. I had to do this on the registry.
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SafeBoot\Minimal
Create new key called MSIServer and change the (Default) value to Service
Open CMD as Administrator and type NET START MSIServer
This allowed me to run the uninstaller and completely remove it without any errors.
Rebooted and then installed it and it worked. I then rebooted and logged into the VPN few times to make sure the issue is 100% resolved.