Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 05-15-2012, 09:49 PM   #1
microcity
Junior Member
microcity began at the beginning.
 
Posts: 2
Karma: 10
Join Date: May 2012
Device: ipad
Calibre slowness as library expands

I started using Calibre about a year ago, and when my library was 10 volumes it was great. Gradually my library has grown to more than 10K volumes, and Calibre has become almost unusably slow. Startup time is more than a minute, and even basic searches can take more than 30 seconds. Does anyone have suggestions on how I can accelerate Calibre?
microcity is offline   Reply With Quote
Old 05-16-2012, 03:23 AM   #2
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Welcome to MobileRead.

Preferences -> Searching -> tick the "Limit the searched metadata" box. Makes an enormous difference to search speed.

There are a number of threads on this subject in the forum if you do a search.
kiwidude is offline   Reply With Quote
 
Enthusiast
Old 05-16-2012, 03:38 AM   #3
mbovenka
Wizard
mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.
 
Posts: 1,155
Karma: 1511856
Join Date: Oct 2007
Location: Almere, The Netherlands
Device: Kobo Aura HD
Quote:
Originally Posted by kiwidude View Post
Preferences -> Searching -> tick the "Limit the searched metadata" box. Makes an enormous difference to search speed.
Or train yourself so use the search specifiers, like 'author:' and 'title:'. That helps a bunch too.
mbovenka is offline   Reply With Quote
Old 05-17-2012, 05:07 AM   #4
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,052
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad1, Kindle3KB
Speeding Up:

1. In external security software, disable virus scanning of official calibre Library folders.

2. Hide the Tag Browser in calibre when not using it directly, particularly before quitting calibre so it won't be open the next time calibre starts up.

3. Disable unnecessary plugins where calibre allows, such as Device Interface plugins for devices you don't use, and any installed but currently unused third-party plugins such as for Metadata Source and User Interface. All enabled plugins take time to initialize at calibre startup. You can enable rarely-used plugins specifically only when you need them.

4. Use as few columns "built from other columns" as possible.

5. Avoid using complex template language or programming particularly where it has to iterate through large numbers of books in the library or restriction, one instance of which occurs at calibre startup.

6. Prefix all searches with the column lookup name, including in saved searches and restrictions. Unprefixed searches in a large library can be time consuming even with Prefs/Searching/"Limit the searched metadata" set to exclude Comments and to include the most useful and simple columns. Also where possible, do intensive search operations within pre-set restrictions ( -- restrictions limit the number of books to be searched through to a subset of the library).

There are probably other performance issues/tips for large libraries that siipped my mind at the moment, or that I haven't learned yet.

Last edited by unboggling; 06-12-2012 at 05:14 AM. Reason: bad assumption
unboggling is offline   Reply With Quote
Old 06-11-2012, 01:23 PM   #5
microcity
Junior Member
microcity began at the beginning.
 
Posts: 2
Karma: 10
Join Date: May 2012
Device: ipad
unboggling... great suggestions! Turning off the plugins was very annoying in the UI, since you have to click through each one. However, the effects are dramatic: program startup is back to a few seconds, and searches are speedy again. Thanks again!
microcity is offline   Reply With Quote
Old 06-11-2012, 08:54 PM   #6
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,052
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad1, Kindle3KB
You're welcome.

Another thing I do is keep my data clean. Metadata Download often brings in various unstandardized data, for example for Publisher various books may have Random House Inc, Random House Inc., Random House for Young Readers, Random House ThisOrThat. Every several months I go through the tag browser cleaning up the variations to one standard name such as "Random House". I do that for all columns where there may be insignificant variations. I don't know if that speeds up searches or if it's just me being obsessive/compulsive.
unboggling is offline   Reply With Quote
Old 06-12-2012, 04:57 AM   #7
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Just to kill off some urban myths here before they perpetuate...

Disabling plugins will have virtually zero impact on startup time (it will at best be a difference measured in milliseconds, certainly not seconds). I have an 11K book library, twenty-something of my own plugins installed and enabled plus meme's Kindle Collections plugin, and all of the standard calibre plugins enabled. My library starts up in 5 seconds.

Secondly plugins will make zero difference to search time - they are not in the slightest bit involved in that process. The best thing you can do for search performance is twofold. Limiting the search terms advice I gave in my previous post above will make enormous difference if your habit is to just type your search words into the search bar without scope prefixes. Alternatively the advice from unboggling of using search prefixes is for optimum results.

I wouldn't think that rationalising your publishers will make any worthwhile difference to search times (particularly for the enormous ongoing and manual effort involved) though I agree for the OCD types it might be a warm fuzzy.

@microcity - if you have found that your library has sped up, I just don't believe plugins were the reason, it will have been something else that has changed. It could be a an OS update, an AV update, a calibre update, a custom column, a disk problem - but I can all but guarantee it will not have been a plugin as the cause.
kiwidude is offline   Reply With Quote
Old 06-12-2012, 05:11 AM   #8
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,052
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad1, Kindle3KB
Huh. Thanks for the clarification, kiwidude. I do have a bad habit of operating under insidious bad assumptions. But now I'll not worry about disabling plugins and keeping my data so clean.
unboggling is offline   Reply With Quote
Old 06-12-2012, 06:05 AM   #9
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
No probs. Speaking from personal experience elsewhere I know that when you have a problem and you start googling for solutions, you can end up grasping at any suggestion you come across in the hope it is the magic answer you were looking for. I would have hated to see future users come across this thread and draw incorrect conclusions like "plugins will slow down calibre so they are bad".

Startup time, search performance and metadata download are three discrete and entirely separate software tasks. They all have individual solutions to resolve any problems with. Disabling slower metadata download plugins like Overdrive (if you had it enabled which is not the default) is a good suggestion to increase the metadata download performance but it will make no difference to calibre startup or search times.

It would certainly be "possible" to write a GUI plugin that poorly impacts calibre's startup times, but as I said above that is certainly not the case with the plugins I have published/use. On a 3-yr old Win 7 64-bit machine with a not particularly fast hard-drive startup time is around 5 secs for 11K books and about 10 seconds for a 33K book library from memory. I am sure I could make that faster by using an SSD and a newer machine but it is good enough for me - and certainly not the "minutes" that some users come on the forums to report. So if you have a library that is taking considerably longer to load then as the saying goes, "these are not the droids you are looking for..."
kiwidude is offline   Reply With Quote
Old 06-12-2012, 08:52 PM   #10
unboggling
by the bootstraps
unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.unboggling ought to be getting tired of karma fortunes by now.
 
Posts: 1,052
Karma: 858115
Join Date: Jan 2011
Location: Southeast US
Device: PRS-T2, Nexus 7, KindleT, iPad1, Kindle3KB
Well, I decided to continue disabling or removing plugins I don't use. Can't hurt.

Last edited by unboggling; 06-13-2012 at 01:41 AM. Reason: Removed some content needing further thought.
unboggling is offline   Reply With Quote
Old 06-14-2012, 03:59 PM   #11
Mythlandia
Reader
Mythlandia got an A in P-Chem.Mythlandia got an A in P-Chem.Mythlandia got an A in P-Chem.Mythlandia got an A in P-Chem.Mythlandia got an A in P-Chem.Mythlandia got an A in P-Chem.Mythlandia got an A in P-Chem.Mythlandia got an A in P-Chem.Mythlandia got an A in P-Chem.Mythlandia got an A in P-Chem.Mythlandia got an A in P-Chem.
 
Posts: 13
Karma: 6184
Join Date: Jun 2009
Location: Celebration, FL
Device: iPad
Disabling the tag browser made a huge difference running my 30k library
Mythlandia is offline   Reply With Quote
Reply

Tags
performance, slow

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
PRS-950 Help for terrible slowness/lagging problems with ebooks sun surfer Sony Reader 5 02-05-2012 11:21 PM
Slowness converting markdown book from the GUI in 7.38 ldolse Calibre 4 01-09-2011 10:21 PM
Slowness of e-book reader mst Calibre 5 12-19-2010 06:11 PM
Overall slowness iharley Calibre 2 07-06-2010 12:29 AM
All Romance expands mtravellerh News 1 07-20-2009 12:55 PM


All times are GMT -4. The time now is 09:23 AM.


MobileRead.com is a privately owned, operated and funded community.