MobileRead Forums - View Single Post - calibredb very slow after large metadata edit

DaltonST · 06-22-2020, 10:39 AM

I believe that differences in RAM utilization and metadata.db file size account for much of what you describe, making it fruitless for others to try to replicate your issues.

Monitor your RAM usage while you do your various tasks. Also monitor the uncompressed size of your metadata.db files.

Optimize the size of your RAM page-file, and defrag your HD.

When you quick-switch from Library A to Library B, Calibre closes A and deletes its cache from RAM. It then loads B into its cache in RAM.

Depending on the size of metadata.db of A compared to B, you could easily notice a difference in initial loading times. Quick-switching to a Library of 40,000 books will always take longer than to a Library of 100 books.

Using calibredb when the GUI is closed obviously uses much less RAM than doing the identical function when the full GUI is open.

When you create a new Library by copying the current Library's structure, and then switch to it, that first switch will be very fast since the new Library is initially empty.

If you are switching among your Libraries just to "look" to see what changed, and not to do any edits or add new books while in the GUI, you might consider running CalibreSpy via a CLI command file (not the Calibre GUI) using calibredb, especially the command file that uses a command-line parameter that invokes a "pre-filter" option so it loads exactly what you are interested in, and not everything. Super-fast. And you can simultaneously view as many Libraries as you wish, always read-only. CS's OP has several flavors of .bat files attached as CLI templates for non-Windows systems.

DaltonST

06-22-2020, 10:39 AM	#7
DaltonST Deviser Posts: 2,265 Karma: 2090983 Join Date: Aug 2013 Location: Texas Device: none	I believe that differences in RAM utilization and metadata.db file size account for much of what you describe, making it fruitless for others to try to replicate your issues. Monitor your RAM usage while you do your various tasks. Also monitor the uncompressed size of your metadata.db files. Optimize the size of your RAM page-file, and defrag your HD. When you quick-switch from Library A to Library B, Calibre closes A and deletes its cache from RAM. It then loads B into its cache in RAM. Depending on the size of metadata.db of A compared to B, you could easily notice a difference in initial loading times. Quick-switching to a Library of 40,000 books will always take longer than to a Library of 100 books. Using calibredb when the GUI is closed obviously uses much less RAM than doing the identical function when the full GUI is open. When you create a new Library by copying the current Library's structure, and then switch to it, that first switch will be very fast since the new Library is initially empty. If you are switching among your Libraries just to "look" to see what changed, and not to do any edits or add new books while in the GUI, you might consider running CalibreSpy via a CLI command file (not the Calibre GUI) using calibredb, especially the command file that uses a command-line parameter that invokes a "pre-filter" option so it loads exactly what you are interested in, and not everything. Super-fast. And you can simultaneously view as many Libraries as you wish, always read-only. CS's OP has several flavors of .bat files attached as CLI templates for non-Windows systems. DaltonST Last edited by DaltonST; 06-22-2020 at 10:41 AM.