Large Library what can be done to speed up Calibre

LadyKate · 08-11-2013, 10:32 AM

I have about 40 000 titles in my library and am in the process of cleaning it up, fixing metadata.

I noticed that Calibre has got a tad slow compared to when the library was at 10 000 books.

My system is a i7 with 32 gig ram using windows 8 and I am wondering, is the speed of Calibre a function of the drive speed? Would it be faster to maintain the library on a solid state drive?

Adoby · 08-11-2013, 11:02 AM

Yes, calibre would be faster with a faster harddrive. But the difference would most likely not be very big. You have plenty of memory and that means that you have plenty of disk cache.

But I suspect that it isn't the IO bound operations that you find slow, like save to disk or conversions. More things like moving from book to book while editing metadata? Or updating metadata fields.

Some things that might help, and are easy to test:

Hide the tag browser. If you have a large library and update metadata, then the tag browser also have to be updated. Can be slow, and unnecessary if you don't need it while fixing metadata.

Remove the formats column. By default this column is populated by reading directly from each book folder to see what formats there are. That can be very slow, especially if you also sort or search on that field.

Here is more about this:

https://www.mobileread.com/forums/sho....php?p=2244745

Also there seems to be a big upgrade of the database backend in the wings. The update is in the latest calibre, but you have to activate the new backend to try it. https://www.mobileread.com/forums/sho...d.php?t=218937

Lady Fitzgerald · 08-11-2013, 10:21 PM

Actually it probably would be faster with a faster HDD. On my XP machine, it took 45 seconds when I first opened calibre after a reboot. When I built my Win 7 machine with an SSD, it takes only two or three seconds to open the first time after a reboot.

My Win 7 notebook came with a HDD. It also needed 45 seconds to open calibre the first time after a reboot. I replaced the spinner. with a SSHD (hybrid drive) and calibre now opens in two or three seconds.

LadyKate · 08-12-2013, 06:19 PM

It is the moving between records and making changes to the tags that is much slower.

There seems to be a lag when I select a record to change or to open that wasn't there when the library was much smaller.

Adoby · 08-12-2013, 06:33 PM

You should notice a difference if you close the tag browser window, when editing metadata.

And if you remove custom columns that depend on other fields, and the formats column, then that should make it faster to move from record to record.

You could also try the new database backend. I belive it is much faster for some of the cases mentioned above. But make sure you have a recent backup. It is not released for "production" yet.

You could also split the library. For instance fiction/non-fiction. Also you could create a new small library for a smaller group of books, like a few authors A-B, and fix the metadata for the books while they are in that small library. Then move them back to the main library.

08-11-2013, 10:32 AM	#1
LadyKate Fanatic Posts: 515 Karma: 1470724 Join Date: Jul 2013 Location: Quebec CA Device: android 4 (samsung tablet and asus tablet)	Large Library what can be done to speed up Calibre I have about 40 000 titles in my library and am in the process of cleaning it up, fixing metadata. I noticed that Calibre has got a tad slow compared to when the library was at 10 000 books. My system is a i7 with 32 gig ram using windows 8 and I am wondering, is the speed of Calibre a function of the drive speed? Would it be faster to maintain the library on a solid state drive?

08-11-2013, 11:02 AM	#2
Adoby Handy Elephant Posts: 1,736 Karma: 26785668 Join Date: Dec 2009 Location: Southern Sweden, far out in the quiet woods Device: Thinkpad E595, Ubuntu Mate, Huawei Mediapad 5, Bouye Likebook Plus	Yes, calibre would be faster with a faster harddrive. But the difference would most likely not be very big. You have plenty of memory and that means that you have plenty of disk cache. But I suspect that it isn't the IO bound operations that you find slow, like save to disk or conversions. More things like moving from book to book while editing metadata? Or updating metadata fields. Some things that might help, and are easy to test: Hide the tag browser. If you have a large library and update metadata, then the tag browser also have to be updated. Can be slow, and unnecessary if you don't need it while fixing metadata. Remove the formats column. By default this column is populated by reading directly from each book folder to see what formats there are. That can be very slow, especially if you also sort or search on that field. Here is more about this: https://www.mobileread.com/forums/sho....php?p=2244745 Also there seems to be a big upgrade of the database backend in the wings. The update is in the latest calibre, but you have to activate the new backend to try it. https://www.mobileread.com/forums/sho...d.php?t=218937 Last edited by Adoby; 08-11-2013 at 11:13 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Calibre opens very slowly with large library?	SilverAce	Calibre	29	09-18-2012 03:11 PM
Best way to get a large Calibre library into the PE library?	Filark	enTourage Archive	0	04-20-2011 10:18 PM
Kindle and Calibre user with problem importing large library into Calibre	pleabargain	Calibre	1	12-07-2010 10:19 AM
Large library/calibre interface	StanByk	Calibre	12	10-19-2009 10:20 AM
How large is your Calibre library?	radius	Calibre	28	10-12-2009 01:14 AM

08-11-2013, 10:21 PM	#3
Lady Fitzgerald Wizard Posts: 2,013 Karma: 251649 Join Date: Apr 2010 Location: Tempe, AZ, USA, Earth Device: JetBook Lite (away from home) + 1 spare, 32" TV (at home)	Actually it probably would be faster with a faster HDD. On my XP machine, it took 45 seconds when I first opened calibre after a reboot. When I built my Win 7 machine with an SSD, it takes only two or three seconds to open the first time after a reboot. My Win 7 notebook came with a HDD. It also needed 45 seconds to open calibre the first time after a reboot. I replaced the spinner. with a SSHD (hybrid drive) and calibre now opens in two or three seconds.

08-12-2013, 06:19 PM	#4
LadyKate Fanatic Posts: 515 Karma: 1470724 Join Date: Jul 2013 Location: Quebec CA Device: android 4 (samsung tablet and asus tablet)	It is the moving between records and making changes to the tags that is much slower. There seems to be a lag when I select a record to change or to open that wasn't there when the library was much smaller.

08-12-2013, 06:33 PM	#5
Adoby Handy Elephant Posts: 1,736 Karma: 26785668 Join Date: Dec 2009 Location: Southern Sweden, far out in the quiet woods Device: Thinkpad E595, Ubuntu Mate, Huawei Mediapad 5, Bouye Likebook Plus	You should notice a difference if you close the tag browser window, when editing metadata. And if you remove custom columns that depend on other fields, and the formats column, then that should make it faster to move from record to record. You could also try the new database backend. I belive it is much faster for some of the cases mentioned above. But make sure you have a recent backup. It is not released for "production" yet. You could also split the library. For instance fiction/non-fiction. Also you could create a new small library for a smaller group of books, like a few authors A-B, and fix the metadata for the books while they are in that small library. Then move them back to the main library.

Advert

Advert