As a follow-up to this: a workaround for this issue may have found its way into the Qt 6.5, 6.4, and 6.3 codebase. Perhaps it will even be released in 6.3.3. The problem was that those wmf dlls were delay-loaded in Qt5 WebEngine, but were not delay-loaded in Qt6 6.2+ WebEngine (not at first anyway).
https://code.qt.io/cgit/qt/qtwebengi...df9b9b3a8daae6
Seems like a pretty straight-forward workaround for those who build their own Qt. I've not tested if it works yet, but hope to soon.