Quote:
Originally Posted by bpothier
UPDATE3: TLDR - "fixed" DB performance issue... Library & DB were on NAS, copied metadata.db to local drive and set ENV VAR "CALIBRE_OVERRIDE_DATABASE_PATH" to point to local file.... metadata update of 14 ePUB dropped to 5 seconds.
Ref: https://www.mobileread.com/forums/sh...d.php?t=294998
Hopefully this helps someone else with DB perf issues...
Details:
Spoiler:
Upgraded to Calibre 5.32. The 14 books that took 8m to update now took 4m. Still an age, but progress. "Only" around 20k books in Library, DB is around 120MB.
Found forum thread - https://www.mobileread.com/forums/sh...d.php?t=294998 - with perf issues around opening/adding/importing. It suggested moving DB file and adding a symlink or setting ENV var. Tried to create windows symlink on NAS share to local disk without success (tried as local admin, changed fsutil R2L symlink option). Setting the ENV variable "CALIBRE_OVERRIDE_DATABASE_PATH=D:\CalibreDB\metad ata.db" does work, though it is less flexible for multiple libraries. After copying DB to local drive, doing FFF metadata update from stored column took... 5 seconds..... Likely will create a few CMD/batch files to set ENV and start Calibre as a workaround. The docs do mention NAS performance issues around locking support, but have not delved into that yet. I don't switch Calibre Libraries very often, so will likely keep it this way for now... Might add a DB copy on startup from local to NAS in the batch script for safety...
I think one issue with the DB on the NAS is how Calibre updates the file. I have "recyclebin" enabled on the NAS and when Calibre does a DB update it creates and deletes a temporary metadata.db-journal file. These seem to have accumulated in the NAS recyclebin as "Copy #1234 of metadata.db-journal" and there are now thousands of them... |
Ah. Putting the library on a NAS is one of the things that's highly recommended one NOT do.
https://manual.calibre-ebook.com/faq...rked-drive-nas
Spoiler:
Do not put your calibre library on a networked drive.
A filesystem is a complex beast. Most network filesystems lack various filesystem features that calibre uses. Some don’t support file locking, some don’t support hardlinking, some are just flaky. Additionally, calibre is a single user application, if you accidentally run two copies of calibre on the same networked library, bad things will happen. Finally, different OSes impose different limitations on filesystems, so if you share your networked drive across OSes, once again, bad things will happen.
Consider using the calibre Content server to make your books available on other computers. Run calibre on a single computer and access it via the Content server or a Remote Desktop solution.
If you must share the actual library, use a file syncing tool like DropBox or rsync instead of a networked drive. If you are using a file-syncing tool it is essential that you make sure that both calibre and the file syncing tool do not try to access the calibre library at the same time. In other words, do not run the file syncing tool and calibre at the same time.
Even with these tools there is danger of data corruption/loss, so only do this if you are willing to live with that risk. In particular, be aware that Google Drive is incompatible with calibre, if you put your calibre library in Google Drive, you will suffer data loss. See this thread for details.