Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 12-14-2008, 06:49 AM   #1
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,071
Karma: 777825
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
Performance with large Libraries?

I am finding that as my library is getting larger, the performance of Calibre seems to be dropping of markedly when adding new books. As I want to add several thousand more this is worrying as it might mean Calibre becomes unusable for a Library of this size.

In case it matters, I am running under Windows XP, with the Calibre library on a network drive, and currently around 1000 books stored in it.

Questions I have are:
- Is this a known issue?
- Is it specific to Windows?
- Is there anything I can do to improve performance?
- Is there anything I can do to diagnose the bottleneck?
itimpi is offline   Reply With Quote
Old 12-14-2008, 09:01 AM   #2
Mitchll
Cave Dweller
Mitchll once ate a cherry pie in a record 7 seconds.Mitchll once ate a cherry pie in a record 7 seconds.Mitchll once ate a cherry pie in a record 7 seconds.Mitchll once ate a cherry pie in a record 7 seconds.Mitchll once ate a cherry pie in a record 7 seconds.Mitchll once ate a cherry pie in a record 7 seconds.Mitchll once ate a cherry pie in a record 7 seconds.Mitchll once ate a cherry pie in a record 7 seconds.Mitchll once ate a cherry pie in a record 7 seconds.Mitchll once ate a cherry pie in a record 7 seconds.Mitchll once ate a cherry pie in a record 7 seconds.
 
Mitchll's Avatar
 
Posts: 220
Karma: 1986
Join Date: Oct 2006
Device: Ipad Sony reader prs-505 prs-900
My library totals over distinct 4000 books, with multiple formats for each. Calibre's performance does not appear to degrade except that sorts and startup are slow so as to account for the volume of the search.

Mitch
Mitchll is offline   Reply With Quote
 
Enthusiast
Old 12-14-2008, 10:38 AM   #3
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,125
Karma: 5381911
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Hmm IIRC adding doesn't involve any operations that scale non-linearly with database size. If you have the source, adding a few

Code:
from time import time
print time.time()
in the _add_books function in gui2/main.py should show you where the slowdown is occurring (you will have to run the GUI using

Code:
calibre-debug -c "from calibre.gui2.main import main; main()"
to see the output
kovidgoyal is offline   Reply With Quote
Old 12-14-2008, 02:46 PM   #4
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,071
Karma: 777825
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
OK - that gives me something to work with.

I thought that with the metadata in a database it should scale reasonably well as the metadata.db file is not that large (only a few megabytes), but as I was having problems I though I would check. I was also concerned that I might be encountering an issue that was Windows specific.

Anyway I will report back when I have more information. It may well be something local to my system and not related Calibre at all.
itimpi is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
For those with large (500+) libraries... ficbot General Discussions 60 12-13-2011 01:51 PM
Scalability and performance blither Calibre 1 08-10-2010 08:31 PM
DR800 Performance jjansen iRex 18 08-03-2010 09:10 PM
DX performance on large (>100MB) pdf files? Blackguard Amazon Kindle 6 06-16-2009 06:57 PM
Library size and performance MrFinch Calibre 4 01-01-2009 04:11 AM


All times are GMT -4. The time now is 07:08 PM.


MobileRead.com is a privately owned, operated and funded community.