Maximum number of books?

[mdibella]

I had thought this would be in the FAQ but I don't see it...apologies if it is there and I just missed it.

My question: Is there a limit as to how many books can be in the Calibre library? Is there a file size limit that applies to the library?

It's awfully easy to get a large library if you have several versions of the books, so I just wondered if I'd eventually hit some size limit.

[itimpi]

As far as I kbnow there is no built in limit. The largest library I have heard of people already having is around 30K books which is quite a large library. I personally currently have about 9K books in a wide variety of formats (averaging around 4 formats per book) - total size is around 14GB.

[kovidgoyal]

There are no limits. All that will happen as your library size grows is performance for searching/sort will decrease.

[al35]

The only limit I run into is the size of individual book. I got a memory error message when trying to add a very large audio book (~2Gb).

[mbovenka]

Originally Posted by itimpi

As far as I kbnow there is no built in limit. The largest library I have heard of people already having is around 30K books which is quite a large library. I personally currently have about 9K books in a wide variety of formats (averaging around 4 formats per book) - total size is around 14GB.

Yep. Just imported 31222 books, total size 11GB. That took a day (literally; it took almost 24 hours), but version 0.6.49 now starts up in 1,5 minutes and searches take < 2 sec.

The older versions were completely unusable on this machine (1.5GB RAM, 1,66GHz Core2Duo T5500) with only ~20K books in them (last time I tried). Startup took more than 20 minutes and searches took minutes. This with a DB created with a humongous batchfile of 'calibredb add'-commands, as importing through the GUI...welll

Great job, Kovid!

[osirisomeomi]

Agreed, it's snappier.

[relder]

I'm running into an issue where my library is 11354 ebooks, and I receive a notification that the application has stopped responding while importing. Is there a trick to the import process?

[theducks]

Originally Posted by relder

I'm running into an issue where my library is 11354 ebooks, and I receive a notification that the application has stopped responding while importing. Is there a trick to the import process?

That may mean the child (of the import) process took longer that Windows thinks it should , Wait it out (at least a equal amount of time to that befor the message) and see if it recovers (usually does).

[ninofarino]

Calibre 2.50 on a PC with the necessary space has been tested with over 180,000 book entries in a single library (of about 1.5 TB) , with on average 2 books per entry, with performance degrading linearly with size (when searching) as expected, but with no functionality problems when performing general library management. It is stable when moving files from one super-library to another. (a super-library, is one having over 100,000 entries. No issues have been found due to size of library, and the current version could easily exceed 200K entries. (I'll keep you posted).

Prior to 2.50 (can't remember the version exactly) the Calibre application would terminate abruptly due to out of memory issues when copying a set of randomly selected books (about 20) for a second time from a library with over 170K entries to another. This problem no longer occurs, even with the 32 bit version of calibre.

Kovid, I must congratulate you. Calibre is a 6 star application in my 5 star grading system. No one else has managed to attain 5 stars. I am happy to assist in any way, and if things go as expected, you may have yourself a benefactor.

[kovidgoyal]

Thanks Always good to hear from someone that has successfully pushed calibre's boundaries.

[mbovenka]

Originally Posted by kovidgoyal

Thanks Always good to hear from someone that has successfully pushed calibre's boundaries.

Another data point: my library is about as large (some 185K boooks), although it's 'only' 250GB because I don't keep multiple formats. No issues, either with the portable version I keep in the library to have a self-contained backup or the 64bit version that's my main install.

The Find Duplicates plugin has reached some limit, though. If I do a Soundex Author + Soundex Title search, calibre starts eating memory until it either blows up (32bit process memory limit reached) or starts swapping my PC to death (64bit version). Using 'Fuzzy' instead of 'Soundex' works without a problem.

But that's not something you can do anything about. Calibre is the best (and best supported) program I have ever used, and I've been playing with this stuff since the early 80's (Sinclair ZX-81 FTW! )

[kovidgoyal]

My first computer was a ZX Spectrum as well, on which I wrote my first program -- which was to generate multiplication tables, so that I had an excuse not to have to memorize them, it output the tables in color and everything

[gridluck]

330K books, so far already 3+ days of indexing, and still counting.. 32GB RAM, only 1.6 of is it used by calibre with 2.4% cpu, on win8.1
What is this program doing there so slow.. ? could definitely optimized for parallelism.. pleASE

[eschwartz]

Don't add so many books at a time. calibre needs to remember them all during the adding books stage... it is much more intensive than simply holding open the database.

[gridluck]

Originally Posted by eschwartz

Don't add so many books at a time. calibre needs to remember them all during the adding books stage... it is much more intensive than simply holding open the database.

calibre is stuck at 100%, with 9% of constant CPU usage, using 2.5GB ram, and little disk activity, what is it doing now?