A while ago I was dealing with excruciatingly slow library open/startup that turned out to be, reproducibly, due to sorting on composite custom columns.
But en route to that discovery I, inconsistently, observed that calibre appeared to be walking through the library on startup -- which might have been a problem I fixed, or might have been a red herring. At different times it seemed to be accessing the cover images, or the .opfs.
My guess is that it wasn't really significant, and that it was at various times either populating the cover browser, or backing up metadata changes to the opfs. (After changing metadata in the database, the changes are queued up to be saved with each book as a low-priority background process, if this isn't completed before calibre is closed, it will resume when calibre restarts.)
But I could well be wrong, and on the off chance I fixed a problem like yours, here's most of what I did:
Upgraded to the most recent version of calibre [always a good place to start]
Set the anti-virus to exclude files in the library folders
Set the anti-virus to exclude the calibre.exe and calibre-parallel.exe processes
(I'm using MSE, same as you mentioned)
Hid the tag browser [this was definitely a big problem in older versions, I'm not convinced it makes much difference now]
Hid the cover browser [I suspect this was a red herring - yes calibre was accessing cover files on startup, but probably not enough to noticeably slow things down]
Defragged the drive and the MFT [This did noticeably improve bulk metadata edits, but I don't think it did much else noticeable]
Stopped sorting on composite columns at startup (as described
here (I mentioned this a few posts ago -- and in my case, this was the real culprit)
I also sped up slow searching by excluding the comments field -- this shouldn't have anything to do with startup, but I'll mention it anyway (Preferences-Searching-Limit the searched metadata)
I was just using Win7's Resource monitor, not logging/debugging so I never really figured out why calibre seemed to be walking through the library, but when it went away I just guessed that it wasn't really significant and that I only noticed it because the library open process was so slow for other reasons.
But maybe the answer to your problem is buried in what I did ....
Also, I see that you mention having a custom column ISBN -- which I'm guessing is a composite column, so if you've ever sorted on it or your Genre column -- that could well be what's causing the slowdown.