How many books in a library?

[PowerOffliner]

Has anybody tested, how many titles can be stored with Microsoft Windows 10 in the latest version of Calibre?
Is there a maximum number of books to keep in one library?
I read about a reduced speed (for access) when the limmit is reached.
And is it possible to join to libraries ?
Thanks for your answers.

[theducks]

Do you have a 6T drive? Books just take drive space in Calibre.
All it keeps in the DB is metadata, so Maxing out your RAM on a 64bit system should handle that for all but insane sized commercial libraries. Yes, there is a bit of slow down, but nothing like we saw back in the days of the PCAT (286 w/2M RAM 40M HD).

(right-click) Copy to Library will merge selected books into the designated Library.
IMHO reasons for separate Libraries:
Security. PG/Adult material kept separate

Significant Metadata (custom field) differences. Wide tables are bloat if they exist only for a small % of the records

Drive space: remember, Calibre can only run on LOCAL attached drives. No NAS or shares for the Libraries used by Calibre.

[BetterRed]

↑ ↑ ↑ ✔️

Where did you read "...about a reduced speed (for access) when the limmit is reached" and when was it written?

I've been using calibre for almost 7 years, my perception is that its gotten faster over time. It's about the same on my current 16GB, i7-7700 as it was on my old 6GB i5-2300 with much slower storage devices and graphics. The only time I ever noticed it was slow was when I put it on a Dell Alien Alpha gaming box, but so was everything else

My biggest library just topped 150,000 books, I have other libraries between 100 and 8,000 books, performance is about the same on all of them, including start up times which is sub 10 seconds. Scrolling the list is pretty much instantaneous, as is searching and virtual library switching.

Tools that access the ebook file contents - e.g. conversion, plugins such as Count pages etc, will have delays, but most of them run in the background which means you can do other things if they're long running. But those delays have nothing to do with the size of the library.

My reason for several libraries is that I want different custom columns for different content - books, audio/video, papers - and there's not a lot of commonality between them.

BR

[ilovejedd]

Originally Posted by theducks

Do you have a 6T drive?

Iirc, biggest capacity HDD right now is 14TB. With $100 sales on 6-10TB drives becoming more frequent this one's easily within the realm of possibility (assuming one has enough content). I shudder to think what performance will be like though.

[theducks]

Originally Posted by ilovejedd

Iirc, biggest capacity HDD right now is 14TB. With $100 sales on 6-10TB drives becoming more frequent this one's easily within the realm of possibility (assuming one has enough content). I shudder to think what performance will be like though.

Yep. I have a 2T Black, and I am forever waiting on the cache to catch up with a lowly core duo 8G RAM. Bigger would probably just be slower.

[Sarmat89]

Originally Posted by theducks

Wide tables are bloat if they exist only for a small % of the records

In sqlite, any empty field takes only 1 byte. That cannot lead to bloat.

[stumped]

the way SSD drive prices are falling, you will be able to have 1Tb or more of SSD.
1TB now is selling in uk at what I paid for 500Gb a year ago !

I already have 500GB SSD C drives on 2 PCs, so my calibre library runs from SSD. but I don't recall it ever being slow back when I had Spinning drives. I put the old 250G ones into USB caddies to use as additional console storage and speed up game loading.

I expect to see cost of of non SSD hard drives go up as sales decline and manufacturing capacity shuts down

[ilovejedd]

Originally Posted by theducks

Yep. I have a 2T Black, and I am forever waiting on the cache to catch up with a lowly core duo 8G RAM. Bigger would probably just be slower.

In fairness, that’s probably just your CPU.

A more modern CPU with plenty of DDR4 RAM and metadata.db on SSD would probably fare much better.

[stumped]

Originally Posted by ilovejedd

In fairness, that’s probably just your CPU.

A more modern CPU with plenty of DDR4 RAM and metadata.db on SSD would probably fare much better.

surely calibre keeps the metadata.db in RAM, so it is always available- if you use quit to mimimised and never actually shut it down ?

[ilovejedd]

Originally Posted by stumped

the way SSD drive prices are falling, you will be able to have 1Tb or more of SSD.
1TB now is selling in uk at what I paid for 500Gb a year ago !

I already have 500GB SSD C drives on 2 PCs, so my calibre library runs from SSD. but I don't recall it ever being slow back when I had Spinning drives. I put the old 250G ones into USB caddies to use as additional console storage and speed up game loading.

Lol, it’s possible to buy a 4TB SSD with how much I paid for a 1TB SSD 5 years ago. Mind, the laptop that got the SSD was very difficult to upgrade (had to remove the keyboard and everything) so I don’t regret going straight to 1TB (I needed it). Laptop’s actually still in use.

With my usage, I need the SSD. I do a lot of bulk edit and library maintenance operations and those are definitely faster on SSD.

[ilovejedd]

Originally Posted by stumped

surely calibre keeps the metadata.db in RAM, so it is always available- if you use quit to mimimised and never actually shut it down ?

Question is how big a database would 6TB+ worth of books require? Besides, if you make any changes, you’re gonna have to hit the database drive eventually.

Mind, Windows does a lot of background disk operations on the OS drive. If one still uses mechanical HDDs for OS, just moving the Calibre library to a different disk can net significant performance gains depending on the task.

https://www.mobileread.com/forums/sh...d.php?t=311440

[yonkyunior]

By the way, is there significant impact when using different type of HDD just calibre use?,
for example WD,

Green HDD: low noise, suitable for home usage, HDD for file downloading, its advantages including lower heat, more quiet, more eco-friendly. Energy-friendly HDD, large capacity storage; adopting IntelliPower technology, the speed can automatically adjust between 5400-7200 rpm. Pros: quiet, low-cost; Cons: low performance, high-delay time, average life expectancy.
Blue HDD: suitable for home usage, high performance and low-cost advantages, 7200 rpm; the disadvantages include the running noise is a bit higher than Green HDD, performance is lower than Black HDD.
Black HDD: High performance, large cache, high-speed. Lias: LS WD Caviar Black. Mainly suitable for enterprise applications that requiring high-end computing such as video/photo editing/rendering and high performance game console, or servers.

Red HDD: Specifically designed for NAS environment that requiring 7days 24 hours continuous working. Western Digital Red HDD are mainly used in 1 to 5 HDDs small and medium scale NAS residential and small enterprise users. Red HDD’s performance is close to the Green HDD, power consumption is low, low noise, can be suitable for continuous working, featuring NASware technology, this technology offers better compatibility, can fit NAS or RAID seamlessly.
Purple HDD: Green HDD AV-GP’s replicate, support ignore cyclic redundancy check (CRC) errors, suitable for video surveillance storage, 24*7 continuous working design.
Ultrastar SATA Series: Data Center HDD, feature up to 10 times the workload rating of desktop drives

especially when new calibre user started and maintain library, especially updating metadata. and use that HDD only for calibre, and do have 2TB books.

[mbovenka]

Originally Posted by ilovejedd

Question is how big a database would 6TB+ worth of books require?

If my 300K/500GB library is any guide, about 7.5 GB

[ilovejedd]

Originally Posted by mbovenka

If my 300K/500GB library is any guide, about 7.5 GB

Curious, large PDF manuals?

My metadata.db is 30MB for a 2.5GB library (if I keep just EPUB format, then library is only ~500MB).

[ilovejedd]

Originally Posted by yonkyunior

By the way, is there significant impact when using different type of HDD just calibre use?,
for example WD,

especially when new calibre user started and maintain library, especially updating metadata. and use that HDD only for calibre, and do have 2TB books.

Not much really. Just a tip, instead of buying a high performance HDD, buy a 4TB HDD or something and short stroke (only format/use the first 2TB). That should help reduce seek times and would probably still cost less than the high performance HDD.