Okay so, I installed a portable on an external hard drive, ran it, made a symlink to my library (though I don't think that actually mattered), added it as the portable's library, and installed the View Manager plugin.
All my customized 'column coloring' and 'column icon' settings transferred over into the portable, along with the VM views I had created on the normal install -- I guess they transferred from the Calibre settings file in my library? (I really am a beginner at all of this...)
But unfortunately switching between VM views in the portable is just as slow as the other... So I guess that says it's not being affected by a different plugin I have installed. But I'm not sure if something else in the transferred-over cusomizations could be the problem.
I've also tried removing all the column color and icon customizations -- no difference in speed. And my columns aren't
that complex, I didn't think... I'll attach a screenshot of them anyway.
I might just be stuck with very slow view-switching, but I appreciate all the input from everyone!