I don't know if you've seen this thread:
https://www.mobileread.com/forums/sho....php?p=2244745
A few ideas on how to speed up calibre up a little.
I don't think that the number of files and folders is a problem at all for ext4. Compared to things like mail servers calibre is tame.
I have my library on a relatively slow Synology 411j NAS, using NFS, but the database on a local SSD and temp in RAM using tempfs. Most of the things you do in calibre, change metadata, search, tag, download metadata and so on, never touch the actual books, only the database. And if it does, most of the work/updates are done in the temp folder, not in the library folders.
Usually the normal disk caches are enough to give a significant speed boost. But it is fun to tinker some more...
Save a few thousand books to disk, and change the format on the fly and/or update metadata using a plugboard. That is a nice stress test... ;D