Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 02-11-2012, 10:25 AM   #1
Motomaggot
I'll always be a newb
Motomaggot began at the beginning.
 
Motomaggot's Avatar
 
Posts: 20
Karma: 10
Join Date: Feb 2012
Device: Kindle Paperwhite & Touch
Pratical Library Size

Building my Library and I've wondered what is the practical library size limits. Is it more an issue of how good a PC you have or does Calibre start to act flaky after a certain point.

Matthew
Motomaggot is offline   Reply With Quote
Old 02-11-2012, 10:30 AM   #2
transmitthis
Addict
transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.
 
transmitthis's Avatar
 
Posts: 285
Karma: 1003542
Join Date: May 2011
Device: Google Nexus 7 16GB
How large are you thinking of ?
transmitthis is offline   Reply With Quote
Old 02-11-2012, 10:34 AM   #3
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,499
Karma: 5567061
Join Date: Aug 2009
Location: The (original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by Motomaggot View Post
Building my Library and I've wondered what is the practical library size limits. Is it more an issue of how good a PC you have or does Calibre start to act flaky after a certain point.

Matthew
It might help if we knew what size you think as 'Big' (we had some folk think it is 50 )

IIRC there are folk with 10,000 books in Calibre.

Columns built from Columns seems to impact speed significantly on libraries of 1000's of books. A simple (custom) column should not impact the speed.

Speed, seems to be the affected property on my lower end (by today's standards ) hardware. RAM, lots of it, helps
theducks is offline   Reply With Quote
Old 02-11-2012, 11:01 AM   #4
Motomaggot
I'll always be a newb
Motomaggot began at the beginning.
 
Motomaggot's Avatar
 
Posts: 20
Karma: 10
Join Date: Feb 2012
Device: Kindle Paperwhite & Touch
Would be over 15,000 books at this point.
Motomaggot is offline   Reply With Quote
Old 02-11-2012, 11:26 AM   #5
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,499
Karma: 5567061
Join Date: Aug 2009
Location: The (original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by Motomaggot View Post
Would be over 15,000 books at this point.
Someone will speak up soon
theducks is offline   Reply With Quote
Old 02-11-2012, 12:32 PM   #6
bgalbrecht
Wizard
bgalbrecht ought to be getting tired of karma fortunes by now.bgalbrecht ought to be getting tired of karma fortunes by now.bgalbrecht ought to be getting tired of karma fortunes by now.bgalbrecht ought to be getting tired of karma fortunes by now.bgalbrecht ought to be getting tired of karma fortunes by now.bgalbrecht ought to be getting tired of karma fortunes by now.bgalbrecht ought to be getting tired of karma fortunes by now.bgalbrecht ought to be getting tired of karma fortunes by now.bgalbrecht ought to be getting tired of karma fortunes by now.bgalbrecht ought to be getting tired of karma fortunes by now.bgalbrecht ought to be getting tired of karma fortunes by now.
 
Posts: 1,018
Karma: 6356407
Join Date: Aug 2007
Device: nook color, nook touch
I think a coworker of mine who loves the darkside might have 10k-20k in his Calibre library. I've got about 7500 in mine, and it's still quite fast. I have to say the worst of it is the effort required to get all the metadata downloaded. I've got about a thousand KDP freebies needing the metadata downloaded (thanks, ATDrake!), and while you can run a mass update, I prefer to check each book so that I can fix the cover if necessary.
bgalbrecht is offline   Reply With Quote
Old 02-11-2012, 12:40 PM   #7
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
It works ok, things get a little more sluggish (startup and search times) but calibre doesn't become "flaky" as such. As theducks says, complex custom columns will be slow as will the tag browser when open, and there are other operations in calibre that are either intentionally throttled (like metadata download) or arguably "should" be (in that they would be stupid to do against 15000 books at once without expecting various issues).

My personal advice (if you need to add metadata, do conversions, resolve duplicates, yada yada) with that many books is to just slowly add books to your library starting with the ones you are most wanting to read and clean as you go. That way calibre will remain quick while you are are doing all the heavy lifting cleanup work and is much more manageable. You will also learn over time about how you personally want to organise your library, naming/tagging conventions, plugins, conversion preferences etc that will be much less intimidating to change your mind about with a smaller sample to work with. After all it is not as if you are going to read them all this year... or the next 10...
kiwidude is offline   Reply With Quote
Old 02-11-2012, 03:11 PM   #8
Noughty
Addict
Noughty is cognizant of many things which escape those who dream only by night.Noughty is cognizant of many things which escape those who dream only by night.Noughty is cognizant of many things which escape those who dream only by night.Noughty is cognizant of many things which escape those who dream only by night.Noughty is cognizant of many things which escape those who dream only by night.Noughty is cognizant of many things which escape those who dream only by night.Noughty is cognizant of many things which escape those who dream only by night.Noughty is cognizant of many things which escape those who dream only by night.Noughty is cognizant of many things which escape those who dream only by night.Noughty is cognizant of many things which escape those who dream only by night.Noughty is cognizant of many things which escape those who dream only by night.
 
Posts: 352
Karma: 103850
Join Date: Apr 2011
Device: Kindle NT
I think that speed is mostly decreased by custom columns made from other columns or other complicated columns. I had those (I even made them in python code with a lot of help which makes it faster in calibre). After deleting them I couldn't believe how much faster calibre became.
Later on I still needed to create one custom column made from other columns behaving like tags and it immediately reduced speed a little. Not as much as before but a still.

I think you can just make the library and if it becomes sluggish or anything just split it into two/three libraries. It's easy to move books between libraries. I have a few (not for speed issues)
Noughty is offline   Reply With Quote
Old 02-12-2012, 07:52 AM   #9
transmitthis
Addict
transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.
 
transmitthis's Avatar
 
Posts: 285
Karma: 1003542
Join Date: May 2011
Device: Google Nexus 7 16GB
I believe Kovid has a 20,000 test library that he uses, so I don't see 15,000 being too much of a problem, apart from being a time vampire that is.
transmitthis is offline   Reply With Quote
Old 02-12-2012, 07:58 AM   #10
melkorbauglir
Enthusiast
melkorbauglir began at the beginning.
 
melkorbauglir's Avatar
 
Posts: 49
Karma: 10
Join Date: Jul 2011
Location: Lahti,Finland
Device: Sony PRS-650,Nokia N-95
Umm..trying to make 100 000 books library right now.....All 8 cores are working at almost max,by look of it its about day job for gaming pc.


25000 entries so far,calibre is still nicely responsive....

Last edited by melkorbauglir; 02-12-2012 at 02:17 PM.
melkorbauglir is offline   Reply With Quote
Old 02-16-2012, 03:59 PM   #11
Krogenar
It's Facebook!
Krogenar began at the beginning.
 
Krogenar's Avatar
 
Posts: 6
Karma: 10
Join Date: Jun 2011
Device: Nook Color
Burgess Meredith

Quote:
Originally Posted by bgalbrecht View Post
I think a coworker of mine who loves the darkside might have 10k-20k in his Calibre library. I've got about 7500 in mine, and it's still quite fast. I have to say the worst of it is the effort required to get all the metadata downloaded. I've got about a thousand KDP freebies needing the metadata downloaded (thanks, ATDrake!), and while you can run a mass update, I prefer to check each book so that I can fix the cover if necessary.
When you reach that many ebooks, you probably have to start wanting to see the world end, just so you would have time to read. Curating a collection that size has really got to eat into your reading time!

Reminds me of the Twilight Zone episode featuring Burgess Meredith (he was the Penguin on the Adam West version of Batman, and Mickey from Rocky) as a librarian who just happens to be inside the vault of a huge library when the Russians launch a Cold War nuclear attack. He survives the apocalypse and goes outside and the world is over, and he gleefully builds stacks of books, one for each year's worth of reading. He's ecstatic at the prospect of having 'time enough' for reading.

Then his glasses fall off and he crushes them underfoot, and staggers off, blindly, into the wasteland.

ANYWAY! What do you do when you've got more books than you're ever likely to be able read? Maybe it's just nice to have so many choices.
Krogenar is offline   Reply With Quote
Old 02-16-2012, 04:24 PM   #12
speakingtohe
Wizard
speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.
 
Posts: 4,596
Karma: 25170848
Join Date: Apr 2010
Device: sony PRS-T1 and T3, Kobo Mini and Aura HD, Tablet
Quote:
Umm..trying to make 100 000 books library right now.....All 8 cores are working at almost max,by look of it its about day job for gaming pc.
I would be interested to know what results you come up with. Hard to imagine 100,000 different titles in one library physical or digital, but I am sure it can be done on Public Domain works alone.

Helen
speakingtohe is offline   Reply With Quote
Old 02-16-2012, 05:19 PM   #13
piper28
Zealot
piper28 ought to be getting tired of karma fortunes by now.piper28 ought to be getting tired of karma fortunes by now.piper28 ought to be getting tired of karma fortunes by now.piper28 ought to be getting tired of karma fortunes by now.piper28 ought to be getting tired of karma fortunes by now.piper28 ought to be getting tired of karma fortunes by now.piper28 ought to be getting tired of karma fortunes by now.piper28 ought to be getting tired of karma fortunes by now.piper28 ought to be getting tired of karma fortunes by now.piper28 ought to be getting tired of karma fortunes by now.piper28 ought to be getting tired of karma fortunes by now.
 
Posts: 106
Karma: 1178328
Join Date: Feb 2011
Device: none
I find that the only thing that becomes an issue for me is that I get a real slowdown when I edit metadata on something, or remove a book. But it's still liveable, I just wouldn't want to do a lot of metadata editing in a large library.
piper28 is offline   Reply With Quote
Old 02-16-2012, 05:26 PM   #14
speakingtohe
Wizard
speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.
 
Posts: 4,596
Karma: 25170848
Join Date: Apr 2010
Device: sony PRS-T1 and T3, Kobo Mini and Aura HD, Tablet
Hmm..

I find removing books to be almost instantaneous, and editing metadata about the same between a 350 book library and a 13,000+ book library. Takes a second or two sometimes but usually less than the time to reposition the mouse for the next edit.

I do have an I7 desktop, but I use slow external hard drives for my ebooks.

Helen
speakingtohe is offline   Reply With Quote
Old 02-16-2012, 07:55 PM   #15
rorschach
Junior Member
rorschach began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Sep 2011
Device: none
I've got 176, 000 books, a lot of them the same

I got up to 10K in Calibre but found it very sluggish so split it into roughly 3 equal parts. Definite speed improvement

rorschach is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
902 Any Speed Decrease with Size of Library? Jozawun PocketBook 2 04-01-2011 04:38 AM
Size DOES Matter -- the Size of your amygdala :) kennyc Lounge 3 12-29-2010 07:54 AM
Full size photo size v. Screensaver size? FF2 Amazon Kindle 5 12-19-2010 11:02 PM
Font Size in Library View Archon Calibre 2 12-13-2010 07:47 PM
Library size and performance MrFinch Calibre 4 01-01-2009 04:11 AM


All times are GMT -4. The time now is 12:33 PM.


MobileRead.com is a privately owned, operated and funded community.