Not certain about this but I seem to remember that ESXi 7.0.3 had an issue with GPU passthrough. We had some fun with AMD video where setting both audio and video to passthrough ended up sometimes working after a restart and sometime not working. You might want to take a look at
VMware vSphere / ESXi 7.0 GA work-around for GPU passthrough issues including disabled-after-reboot bug and UI bug. I think this is where my co-worker found the basis for our workaround.