Calibre caches recently read books in the user %appdata% folder. If that resides on an SSD then it's competing for what is probably a limited resource, as a consequence cached data has to be flushed.
If you have a hard disk, try relocating calibre's cache via the CALIBRE_CACHE_DIRECTORY
Environment variable
BR