The VPN is for them to protect their network from the external device, it's a one way restriction. They don't need the VPN on to access the external device, Windows have plenty of tools to allow them to do that, they just need the device to be on. They can view, install, access logs and files.