[kovidgoyal]

The next build will have a disk cache for cover thumbnails. This persists between calibre restarts and stores the resized thumbnails for all covers.

This is fast enough that we can go back to rendering the covers live during scrolling, and there's no need to maintain a large in memory cache of the covers (though you still can if you want to).

EDIT: This means that the very first time you use the cover grid, it will be slow when scrolling. But after that it should be fast (of course if you change the cover size then it will be slow again the first time).