Just wanted to pop back into this thread and say, all delays in loading VM Views are, indeed, totally gone now (at least when I'm not relying on Action Chains to change the VM view). After updating to Calibre 5.39.0 and installing View Manager 1.7.1, I set each VM view to switch to a specific virtual library, and now CTRL+Tabbing between them instantly loads the correct VL and VM column layout. Yay!!
Using Action Chains (set to change VM views when clicking on a VL tab) does reintroduce some delay. As chaley mentioned earlier, apparently that delay is intentional to "improve performance." So I'm trying to rely less on clicking a tab and having Action Chains change the VM view, and more on using CTRL+Tab to switch VLs/VM views.
I kind of wish there was a shortcut to load the previous VM view, in addition to CTRL+Tab which loads "next view," but that's a not a huge deal.
Thanks again to everyone who helped figure out what was up with those delays. It makes switching between VM views so much smoother.