The days of the simple rotary A - B VGA switch are long gone.
(there were 3 pins in the VGA connector, that the smarter switches maintained, even when switched away)
With Computers doing active Auto-detect (allows add or remover Keyboard or Mice or other USB devices), it detects the instant change.
As with anything AUTO, MURPHY still rules.
You should see what happens when you want to use your TV as an extra Monitor.
