Quote:
Originally Posted by ownedbycats
I know this is an old thread, but I was curious about it too, mostly the hard limit of books the Calibre database can hold. 
Calibre uses a sqlite database, and according to the SQLite documentation, the theoretical maximum number of rows in a table is 2*64 (no superscript?), which is....a very big number. Though you'll reach the 140 terabyte database size limit long before that number. |
Havent run into it yet.. Over 60,000 directories and 260000 items in library. Now running on a 2tb intel M3 drive at about 1000 mb a second. To be honest I don't really see much performance increase over the old 1tb samsung ssd running at 500ish MB a second in searches etc.. Though that might be due to System IO bottlenecks either hardware or OS. I don't feel as if I am getting the IO performance that my underlying drives should be getting. Average author search in my library takes 4 to 8 seconds to come back. A bulk convert of a bunch of books seems to average 2 to 15 seconds per book. Though doing it as part of syncing some books to my kindle takes much much longer. If I convert and then sync it a much shorter process. Starting calibre after I boot system takes a minute to a minute and a half before it's ready to use. I think this lag is mostly due to the initial look at the directory the library is in because of the vast amount of directories in it. This is an underlying OS issues not a calibre issue. I use the built in server which works well, I am not sure if it affects start up time if it is enabled. I am not sure on searches why it takes as long as it does to come back with a reply even if there are 260000 books in library. I know I deal with big databases that make that look really really small that don't have that kind of lag on much slower infrastructure. I have just assumed that it has to do with the type of database used that is embedded in calibre. However performance is good enough that I am not frustrated using it. And over the years it has gotten incrementally and significantly better performance wise over time as it has been improved. At one time with less than half the books in it searches might take 20ish seconds compared to now.
My current system though fast is 10 years old now. It's fast enough that I haven't been able to justify the 1000 dollars or so to upgrade to one of the new AMD's systems with more cores and lots of PCIE lanes. Maybe late this year or next year.
Remember to every once in a while kick a donation through paypal to the developers. This is an awesome program that I have used since pre 2012. I try to send some money every year or two to them. Same with other open source programs I use on a regular basis. I would hate to see Calibre disappear like so many other programs I have used in the past.
Update 1/2023 see new post
from me below. significant performance increase since this with new hardware and 30% ish larger library