Is there a limit to the number of books Calibre can handle?

[audeojude]

Update.

over 325,000 books in library
77,000 directories in library root
1.3 million files in directory
640ish meg database file.

running on Ryzen 7 2700x cpu with 32 gig ram and a crucial 4t nvme at 3500mbs
opening calibre in empty library 3 seconds
opening on this library 18 seconds
search 2 seconds.

you can see my post on my old system above from june of 2020 to contrast it. Defiantly faster than it was with new hardware and upgraded nvme. Actually I think it is fastest its ever been. Win for throwing hardware at it.

[audeojude]

Originally Posted by georgestancliffe

And I thought my E-Book collection was BIG. I have about 7,000 E-books in my laptop. Most of them were downloaded from Gutenberg.org, for free. If I read one book each day, it would take me about 20 years to read them all!

I'm glad to hear that the CALIBRE E-book reader can handle over 250,000-book super-libraries!

as new releases come out and in conjunction with faster hardware calibre is getting much more usable.. not that it wasn't at it's slowest for me. I suppose if I was using a spinney hard drive at the size it is now I would be very very unhappy with search times and start up times.

lol... pretty funny having my library called a super library. My physical library of books is in excess of 10,000 books. about 5,000 paperbacks and maybe 6000 hardbacks. mix of non fiction and fiction.. probably 4000 non fiction hardbacks.
we actually purchased a 48ft insulated tractor trailer and put in bamboo floors and ac along with shelves for all the books as the house foundations were not happy with the weight when all of them were lining our walls.

Digital just allowed me to go even crazier

I read about 500 - 700 books a year I keep telling myself as an obsessive hobby its better than doing drugs...

[Doum]

Yes it can handle a lot but you need the 64bits version, a good Disk (RAM) and a good CPu

[DNSB]

The most I remember one poster mentioning was having close to 1,000,000 items in a calibre library. The main complaints were that startup times were longer and most operations were slower. The computer had 8GB of RAM and a spinning rust hard drive.

[JSWolf]

Originally Posted by DNSB

The most I remember one poster mentioning was having close to 1,000,000 items in a calibre library. The main complaints were that startup times were longer and most operations were slower. The computer had 8GB of RAM and a spinning rust hard drive.

That is going to be very slow.

[DNSB]

Most people seemed to agree with you. Suggesting an upgrade in memory and CPU and upgrading to a NVMe SSD was the consensus. Reducing the number of custom columns wa also popular.

[JSWolf]

How do you get 1,000,000 (roughly) eBooks without major piracy?

[mbovenka]

Originally Posted by JSWolf

That is going to be very slow.

The 'spinning rust' bit actually matters less than you'd think. Calibre loads the metadata db into RAM at startup (so startup would be a bit slower), and searches and such are done on the in-memory copy. So only bulk updates would be slower, and you're not likely to do all that many of those.

RAM and CPU are more important, and 8GB RAM probably is pushing it with 1M database entries.

[JSWolf]

Originally Posted by mbovenka

The 'spinning rust' bit actually matters less than you'd think. Calibre loads the metadata db into RAM at startup (so startup would be a bit slower), and searches and such are done on the in-memory copy. So only bulk updates would be slower, and you're not likely to do all that many of those.

RAM and CPU are more important, and 8GB RAM probably is pushing it with 1M database entries.

It's going to be caching quite a lot and given that it's a hard drive, it's most likely going to be slow. It would be best to break up the library into two or more smaller libraries.

[BetterRed]

Originally Posted by DNSB

. . . close to 1,000,000 items in a calibre library.

Originally Posted by JSWolf

How do you get 1,000,000 (roughly) eBooks without major piracy?

read before you react.

BR

[Quoth]

Entirety of the main PD ebook places in the main formats gets you to nearly 500,000 items.
Then there are PDF manuals, datasheets, service info, old magazines, scans on wayback machine, websites converted.
See http://www.americanradiohistory.com for technical magazines in UK ones since 1920s.

Hmm, now that I've got fibre and Unlimited is now really 3000 Gbyte instead of 350 GByte per rolling 30 days…

It's a bit mad and being immortal would be a help in consumption, but we can't assume piracy.

[Quoth]

Originally Posted by DNSB

Most people seemed to agree with you. Suggesting an upgrade in memory and CPU and upgrading to a NVMe SSD was the consensus. Reducing the number of custom columns wa also popular.

But backup important with SSD, not advance warning of total fail.
I have a hybrid 512 G SSD (NVMe PCIe) and 4,000 G SATA 3.5" HDD, with var and home on HDD partitions (16 G RAM). Calibre is very much faster than the 500 GB 5400 RPM 2.5" HDD on laptop (8G RAM).

But only 5260 approx titles, some in multiple formats. About 5 or 6 seconds to load 1st time, about 3 seconds to load on subsequent times on the Hybrid system, an Intel® Core™ i7-6700 CPU @ 3.40GHz × 8 with 4K screen. The Laptop is a Lenovo E460 i5 with HD screen, about 6 years old. It's fine too.
Both Linux Mint 20.1 with Mate. Win 7 on the laptop was far slower.

At the time I got the hybrid desktop (on a UPS), the 512 G NVMe was much better price/performance than 1T NVMe and the SATA SSD seem a lot slower.
My eldest grandson got a S/H Dell which had 1T SATA HDD but it also had a PCIe NVMe slot, so his dad paid for 512G and it's also setup now with OS on SSD but var and home mapped to two partitions on the HDD. Boot time reduced from 80s to 15s.

[DoctorOhh]

Originally Posted by Quoth

But backup important with SSD, not advance warning of total fail.

You say this as if performing backups isn't essential with all drives.

[BetterRed]

Originally Posted by Quoth

Then there are PDF manuals, datasheets, service info, old magazines, scans on wayback machine, websites converted.

Plus the countless texts that have been emitted by tens of thousands of legislatures, courts, government departments and quango's for centuries. Some of the more important/interesting texts have been digitised.

BR

[ownedbycats]

According to Google, Project Gutenberg has "over 60,000" eBooks.