Calibre excruciatingly slow at adding new books or editing metadata

drose25 · 02-21-2018, 10:50 PM

I used Calibre a few years ago and it was fairly responsive. Installed the 64-bit version tonight on my new PC and it's moving really slowly for some reason. Adding books is taking forever and even editing metadata takes awhile to save changes. Is there a setting I could be overlooking? The program is using virtually no CPU or memory resources, so I don't think the PC is bottlenecking it.

kovidgoyal · 02-21-2018, 10:55 PM

Either you have your calibre library on a very slow drive, or your antivirus is interfering with filesystem operations.

drose25 · 02-22-2018, 12:13 AM

Thanks. I moved my Calibre library to an SSD and it's loads faster, even though the books I'm importing remain on the same standard drive I had the library on earlier. I guess it just wasn't handling importing and storing on the same drive well.

Thanks for the tip! This is much more usable now.

Sarmat89 · 02-22-2018, 08:47 AM

For some reason, Calibre expands every file you add in your temporary folder. It is a very excessive operation.

Adoby · 02-24-2018, 10:51 PM

I assume that calibre is reading metadata and checking that the file is a valid book. Hard to do without unpacking. Calibre also may unzip, modify and zip again when you save or convert books. Hard to avoid if the book is to be updated...

The easiest way to speed up the unzip/zip is to make sure that the tmp folder is on the fastest HDD you have. Even faster is to use a RAM disk for tmp. In Windows you have to install some 3rd party software. In Linux it is built in (tmpfs) and you just add a line to fstab.

Only install a RAM disk if you have plenty of RAM in your computer.

You can specify the tmp directory used by Calibre using an environment variable, if you don't want to use the default.

https://manual.calibre-ebook.com/cus...ment-variables

vonronge · 04-23-2018, 03:58 PM

[[I had to re-register to post since my account seems to be gone. Serves me right, I haven't logged in for six or seven years.]]

I created an environment variable pointing to a RAM disk.

This has increased the speed of adding books to the library a great deal.

To add 10,000 books to a library with over 300,000 books was taking a whole day.

Closing the tag browser, setting the processes # to 14 (Ryzen 7) and setting the temp directory as a ram disk has reduced this to about 5 hours.

Unfortunately the books and the library are on the same directory, might change this in future to speed this up.

I should probably figure out how to move the "metadata.db" and "metadata.db-journal" files to a faster directory, but it seems to require a batch script?

When adding 10,000 books, the size of the \calibre_ft_w80\0gszgc_add_books directory grows to about 2.5 gb

Keeping track of the number of files in this directory is (for me) a much more satisfying way to keep track of the task than the % completion bar, since I know how many are left to do. Purely psychological.

02-21-2018, 10:50 PM	#1
drose25 Junior Member Posts: 5 Karma: 10 Join Date: May 2012 Device: Sony PRS-600	Calibre excruciatingly slow at adding new books or editing metadata I used Calibre a few years ago and it was fairly responsive. Installed the 64-bit version tonight on my new PC and it's moving really slowly for some reason. Adding books is taking forever and even editing metadata takes awhile to save changes. Is there a setting I could be overlooking? The program is using virtually no CPU or memory resources, so I don't think the PC is bottlenecking it.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Help Editing Metadata of New Books While Adding Them	Kevin1a	Library Management	7	06-16-2016 05:40 AM
Programmatically adding/editing metadata	northernbio	Library Management	6	12-29-2012 05:59 PM
Calibre is incredibly slow editing metadata	McCamel	Calibre	9	08-30-2012 05:22 PM
Adding only mobi books to calibre to change metadata	Carolina Troll	Calibre	6	02-11-2012 07:27 PM
what calibre does when adding books? (metadata)	jomaweb	Calibre	1	02-06-2010 12:10 PM

02-21-2018, 10:55 PM	#2
kovidgoyal creator of calibre Posts: 43,843 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Either you have your calibre library on a very slow drive, or your antivirus is interfering with filesystem operations.

02-22-2018, 12:13 AM	#3
drose25 Junior Member Posts: 5 Karma: 10 Join Date: May 2012 Device: Sony PRS-600	Thanks. I moved my Calibre library to an SSD and it's loads faster, even though the books I'm importing remain on the same standard drive I had the library on earlier. I guess it just wasn't handling importing and storing on the same drive well. Thanks for the tip! This is much more usable now.

02-22-2018, 08:47 AM	#4
Sarmat89 Evangelist Posts: 482 Karma: 2267928 Join Date: Nov 2015 Device: none	For some reason, Calibre expands every file you add in your temporary folder. It is a very excessive operation.

02-24-2018, 10:51 PM	#5
Adoby Handy Elephant Posts: 1,736 Karma: 26785668 Join Date: Dec 2009 Location: Southern Sweden, far out in the quiet woods Device: Thinkpad E595, Ubuntu Mate, Huawei Mediapad 5, Bouye Likebook Plus	I assume that calibre is reading metadata and checking that the file is a valid book. Hard to do without unpacking. Calibre also may unzip, modify and zip again when you save or convert books. Hard to avoid if the book is to be updated... The easiest way to speed up the unzip/zip is to make sure that the tmp folder is on the fastest HDD you have. Even faster is to use a RAM disk for tmp. In Windows you have to install some 3rd party software. In Linux it is built in (tmpfs) and you just add a line to fstab. Only install a RAM disk if you have plenty of RAM in your computer. You can specify the tmp directory used by Calibre using an environment variable, if you don't want to use the default. https://manual.calibre-ebook.com/cus...ment-variables

04-23-2018, 03:58 PM	#6
vonronge Junior Member Posts: 2 Karma: 10 Join Date: Apr 2018 Device: Kobo Aura HD/Kindle	[[I had to re-register to post since my account seems to be gone. Serves me right, I haven't logged in for six or seven years.]] I created an environment variable pointing to a RAM disk. This has increased the speed of adding books to the library a great deal. To add 10,000 books to a library with over 300,000 books was taking a whole day. Closing the tag browser, setting the processes # to 14 (Ryzen 7) and setting the temp directory as a ram disk has reduced this to about 5 hours. Unfortunately the books and the library are on the same directory, might change this in future to speed this up. I should probably figure out how to move the "metadata.db" and "metadata.db-journal" files to a faster directory, but it seems to require a batch script? When adding 10,000 books, the size of the \calibre_ft_w80\0gszgc_add_books directory grows to about 2.5 gb Keeping track of the number of files in this directory is (for me) a much more satisfying way to keep track of the task than the % completion bar, since I know how many are left to do. Purely psychological.