Pratical Library Size

[Motomaggot]

Building my Library and I've wondered what is the practical library size limits. Is it more an issue of how good a PC you have or does Calibre start to act flaky after a certain point.

Matthew

[transmitthis]

How large are you thinking of ?

[theducks]

Originally Posted by Motomaggot

Building my Library and I've wondered what is the practical library size limits. Is it more an issue of how good a PC you have or does Calibre start to act flaky after a certain point.

Matthew

It might help if we knew what size you think as 'Big' (we had some folk think it is 50 )

IIRC there are folk with 10,000 books in Calibre.

Columns built from Columns seems to impact speed significantly on libraries of 1000's of books. A simple (custom) column should not impact the speed.

Speed, seems to be the affected property on my lower end (by today's standards ) hardware. RAM, lots of it, helps

[Motomaggot]

Would be over 15,000 books at this point.

[theducks]

Originally Posted by Motomaggot

Would be over 15,000 books at this point.

Someone will speak up soon

[bgalbrecht]

I think a coworker of mine who loves the darkside might have 10k-20k in his Calibre library. I've got about 7500 in mine, and it's still quite fast. I have to say the worst of it is the effort required to get all the metadata downloaded. I've got about a thousand KDP freebies needing the metadata downloaded (thanks, ATDrake!), and while you can run a mass update, I prefer to check each book so that I can fix the cover if necessary.

[kiwidude]

It works ok, things get a little more sluggish (startup and search times) but calibre doesn't become "flaky" as such. As theducks says, complex custom columns will be slow as will the tag browser when open, and there are other operations in calibre that are either intentionally throttled (like metadata download) or arguably "should" be (in that they would be stupid to do against 15000 books at once without expecting various issues).

My personal advice (if you need to add metadata, do conversions, resolve duplicates, yada yada) with that many books is to just slowly add books to your library starting with the ones you are most wanting to read and clean as you go. That way calibre will remain quick while you are are doing all the heavy lifting cleanup work and is much more manageable. You will also learn over time about how you personally want to organise your library, naming/tagging conventions, plugins, conversion preferences etc that will be much less intimidating to change your mind about with a smaller sample to work with. After all it is not as if you are going to read them all this year... or the next 10...

[Noughty]

I think that speed is mostly decreased by custom columns made from other columns or other complicated columns. I had those (I even made them in python code with a lot of help which makes it faster in calibre). After deleting them I couldn't believe how much faster calibre became.
Later on I still needed to create one custom column made from other columns behaving like tags and it immediately reduced speed a little. Not as much as before but a still.

I think you can just make the library and if it becomes sluggish or anything just split it into two/three libraries. It's easy to move books between libraries. I have a few (not for speed issues)

[transmitthis]

I believe Kovid has a 20,000 test library that he uses, so I don't see 15,000 being too much of a problem, apart from being a time vampire that is.

[melkorbauglir]

Umm..trying to make 100 000 books library right now.....All 8 cores are working at almost max,by look of it its about day job for gaming pc.


25000 entries so far,calibre is still nicely responsive....

[Krogenar]

Originally Posted by bgalbrecht

I think a coworker of mine who loves the darkside might have 10k-20k in his Calibre library. I've got about 7500 in mine, and it's still quite fast. I have to say the worst of it is the effort required to get all the metadata downloaded. I've got about a thousand KDP freebies needing the metadata downloaded (thanks, ATDrake!), and while you can run a mass update, I prefer to check each book so that I can fix the cover if necessary.

When you reach that many ebooks, you probably have to start wanting to see the world end, just so you would have time to read. Curating a collection that size has really got to eat into your reading time!

Reminds me of the Twilight Zone episode featuring Burgess Meredith (he was the Penguin on the Adam West version of Batman, and Mickey from Rocky) as a librarian who just happens to be inside the vault of a huge library when the Russians launch a Cold War nuclear attack. He survives the apocalypse and goes outside and the world is over, and he gleefully builds stacks of books, one for each year's worth of reading. He's ecstatic at the prospect of having 'time enough' for reading.

Then his glasses fall off and he crushes them underfoot, and staggers off, blindly, into the wasteland.

ANYWAY! What do you do when you've got more books than you're ever likely to be able read? Maybe it's just nice to have so many choices.

[speakingtohe]

Umm..trying to make 100 000 books library right now.....All 8 cores are working at almost max,by look of it its about day job for gaming pc.

I would be interested to know what results you come up with. Hard to imagine 100,000 different titles in one library physical or digital, but I am sure it can be done on Public Domain works alone.

Helen

[piper28]

I find that the only thing that becomes an issue for me is that I get a real slowdown when I edit metadata on something, or remove a book. But it's still liveable, I just wouldn't want to do a lot of metadata editing in a large library.

[speakingtohe]

Hmm..

I find removing books to be almost instantaneous, and editing metadata about the same between a 350 book library and a 13,000+ book library. Takes a second or two sometimes but usually less than the time to reposition the mouse for the next edit.

I do have an I7 desktop, but I use slow external hard drives for my ebooks.

Helen

[rorschach]

I've got 176, 000 books, a lot of them the same

I got up to 10K in Calibre but found it very sluggish so split it into roughly 3 equal parts. Definite speed improvement