07-09-2013, 01:52 AM | #1 |
Junior Member
Posts: 4
Karma: 10
Join Date: Jul 2013
Device: windows 7
|
why is calibre slow on batch operations?
can someone tell me how to speed up calibres add to library operation. it should not take 10 minutes to add 20 books totaling 12 MB the sheer speed and power of modern computers can move this amount of data in under a second so why is calibre dragging ass?
|
07-09-2013, 07:25 AM | #2 |
null operator (he/him)
Posts: 20,544
Karma: 26944418
Join Date: Mar 2012
Location: Sydney Australia
Device: none
|
If you have Preferences->Add Books->Automatically convert books to the current output format checked then its converting each book - and conversions can sometimes be slow - especially on large PDFs.
BR |
07-09-2013, 08:12 AM | #3 |
Handy Elephant
Posts: 1,736
Karma: 26785668
Join Date: Dec 2009
Location: Southern Sweden, far out in the quiet woods
Device: Thinkpad E595, Ubuntu Mate, Huawei Mediapad 5, Bouye Likebook Plus
|
Calibre not only has to move the data, depending on your settings calibre also has to explode/unzip the books an examine the contents. And perhaps convert and zip again. And it has to do this in a way that ensures that the calibre library doesn't loose data.
Calibre sometimes automatically does a lot of very heavy lifting, depending on the preferences set. It is without any doubt the most demanding program I have installed on my computer, perhaps with the exception of Handbrake for video conversion. But Handbrake doesn't put a load on the I/O like calibre does, mainly cpu. |
07-09-2013, 10:11 AM | #4 |
Wizard
Posts: 2,016
Karma: 13471689
Join Date: Oct 2007
Location: Almere, The Netherlands
Device: Kobo Sage
|
I don't find book import all that slow. But what I have wondered about is why batch metadata updates are so much slower than single-book metadata updates.
Say, correcting a title or author sort for a few books. Do it 1 book at a time, and they take less than a second each. Do even two at a time (thus using the batch metadata update screen) and it takes far longer than (N x <1s). I can't really explain that one. |
07-09-2013, 12:15 PM | #5 |
Handy Elephant
Posts: 1,736
Karma: 26785668
Join Date: Dec 2009
Location: Southern Sweden, far out in the quiet woods
Device: Thinkpad E595, Ubuntu Mate, Huawei Mediapad 5, Bouye Likebook Plus
|
Try to minimize the tag browser to the left. I have have it minimized all the time, except when I actually need it.
Usually when you update metadata only some records in the database is changed. But when you update authors or title, then the books have to be moved to a new folder in the calibre library. Also when the books are moved they are really first copied to the new place and then deleted at the old. Not just renamed. The reason is that this reduces the chance of data corruption if the process is interrupted for some reason. If only one book is updated, then the disc caches can most likely be fully used to speed it up. But maybe not as likely if several books are updated. More RAM will most likely help. |
07-09-2013, 01:22 PM | #6 |
Junior Member
Posts: 4
Karma: 10
Join Date: Jul 2013
Device: windows 7
|
so if it is all the extra work calibre is doing how do i dumb it down a bit to streamline the process because i really don't care what format the book is in as long as it is supported.
on a side note will copying an entire calibre library from one computer to another fresh install bring the preferences or is that stored in appdata seperatly |
07-09-2013, 01:26 PM | #7 | |
Wizard
Posts: 4,812
Karma: 26912940
Join Date: Apr 2010
Device: sony PRS-T1 and T3, Kobo Mini and Aura HD, Tablet
|
Quote:
Best to copy both. Helen |
|
07-09-2013, 01:30 PM | #8 | |
Wizard
Posts: 4,812
Karma: 26912940
Join Date: Apr 2010
Device: sony PRS-T1 and T3, Kobo Mini and Aura HD, Tablet
|
Quote:
Helen |
|
07-09-2013, 01:46 PM | #9 | ||
US Navy, Retired
Posts: 9,864
Karma: 13806776
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Nexus 7
|
Quote:
|
||
07-09-2013, 02:31 PM | #10 | |
Wizard
Posts: 4,812
Karma: 26912940
Join Date: Apr 2010
Device: sony PRS-T1 and T3, Kobo Mini and Aura HD, Tablet
|
Quote:
Kovid Goyal explained to me that the tag browser took a lot of time to update every time a field was changed. Even with the tag browser off though it takes a long time to do a bulk metadata change of even one field in two records, so there is something else in play here. Helen |
|
07-09-2013, 02:37 PM | #11 | |
US Navy, Retired
Posts: 9,864
Karma: 13806776
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Nexus 7
|
Quote:
|
|
07-10-2013, 04:37 AM | #12 | |
Wizard
Posts: 2,016
Karma: 13471689
Join Date: Oct 2007
Location: Almere, The Netherlands
Device: Kobo Sage
|
Quote:
It does seem database size related, because the effect is a lot less when I do it on my small library (which is about a sixth the size of my main one) |
|
07-10-2013, 04:48 AM | #13 | |
Wizard
Posts: 2,016
Karma: 13471689
Join Date: Oct 2007
Location: Almere, The Netherlands
Device: Kobo Sage
|
Quote:
It's database size related, too; the effect is a lot less if your library is smaller. |
|
07-10-2013, 06:04 AM | #14 |
creator of calibre
Posts: 43,835
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Single updates do not cause a resort, bulk updates do. Sorting in large libraries is slow.
|
07-10-2013, 08:06 AM | #15 |
Wizard
Posts: 2,016
Karma: 13471689
Join Date: Oct 2007
Location: Almere, The Netherlands
Device: Kobo Sage
|
|
Thread Tools | Search this Thread |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Batch Converting Ebooks with Calibre and Python | jmeb | Conversion | 17 | 10-20-2018 11:16 AM |
Batch transfer books from Kindle to Calibre | engt | Library Management | 7 | 08-21-2014 06:54 AM |
PRS-T1 questions on collections and slow operations | Jade Aislin | Sony Reader | 9 | 08-14-2012 01:49 PM |
Calibre Portable batch file. | soleilos | Calibre | 2 | 08-20-2011 11:52 AM |
calibre - very slow conversion, very slow on PRS | cremofix | Calibre | 3 | 06-10-2009 04:21 PM |