Register Guidelines E-Books Today's Posts Search

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 10-18-2019, 01:31 AM   #1
Dorian67
Member
Dorian67 began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Oct 2019
Device: Amazon Kindle
Is it possible to have the books (Library) an the metadata.db on different drives?

Hello,

I have problems with the search in my library (more then 100.000) books.
It takes 2-3 minutes to get the result.
The library is actual on a external USB3 Drive.
So the Calibre search in the metadata.db (I think), I would like to have it on a internal SSD, but the books should stay on the external drive.

Is that possible?

Regards Dorian
Dorian67 is offline   Reply With Quote
Old 10-18-2019, 01:37 AM   #2
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 21,725
Karma: 29711016
Join Date: Mar 2012
Location: Sydney Australia
Device: none
See 4th item in ==>> here

BR
BetterRed is offline   Reply With Quote
Advert
Old 10-18-2019, 01:45 AM   #3
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 21,725
Karma: 29711016
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by BetterRed View Post
See 4th item in ==>> here

BR
On Windows, if you have multiple libraries you could use symlinks - e.g.

f:\calibre\library1\metadata.db ==>> c:\calibre\library1.db
f:\calibre\library2\metadata.db ==>> c:\calibre\library2.db

Not sure if that would work MacOS and Linux.
BetterRed is offline   Reply With Quote
Old 10-18-2019, 01:50 AM   #4
Dorian67
Member
Dorian67 began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Oct 2019
Device: Amazon Kindle
Thank you for the very fast answer!

- Is "CALIBRE_OVERRIDE_DATABASE_PATH" the right one?

- To you work with "symlinks" (I run Calibre on a windows PC)

Dorian
Dorian67 is offline   Reply With Quote
Old 10-18-2019, 02:39 AM   #5
Dorian67
Member
Dorian67 began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Oct 2019
Device: Amazon Kindle
Ok I moved the db with symlinks, but the search is still slowly.
I have to search for an other solution.
Dorian67 is offline   Reply With Quote
Advert
Old 10-18-2019, 03:39 AM   #6
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 21,725
Karma: 29711016
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Caveat - calibre reads the database into memory when a library is opened, so relocating it won't make much, if any, difference.

Are you sure the symlinks are correct, should look like this

Click image for larger version

Name:	Annotation 2019-10-18 180435.jpg
Views:	85
Size:	213.6 KB
ID:	174310

if you only have one library, move the metadata.db to the SSD and use the Environment variable

Click image for larger version

Name:	Annotation 2019-10-18 182354.jpg
Views:	71
Size:	250.3 KB
ID:	174311

Some people say it makes a difference if you put the database on a RAM drive - but I've not tried it, nor am I convinced it would be that much faster.

BR
BetterRed is offline   Reply With Quote
Old 10-18-2019, 07:04 AM   #7
mbovenka
Wizard
mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.
 
Posts: 2,079
Karma: 14079267
Join Date: Oct 2007
Location: Almere, The Netherlands
Device: Kobo Sage
Quote:
Originally Posted by BetterRed View Post
Caveat - calibre reads the database into memory when a library is opened, so relocating it won't make much, if any, difference.
Yep. All readonly activities get done on the in-RAM DB image, so searches and such are not impacted at all by the speed of your storage.

My metadata DB is about 600MB, and when I migrated to SSD Calibre's startup time decreased by a few seconds, and anything that updates the database became faster (sometimes by a lot: 'calibredb backup_metadata', which obviously writes a lot of very small files, became about 15 times faster), but search speed, where the OP's issue is, did not change.

So, to the OP: don't bother. If searches take too long, the bottleneck is not your disk, but CPU and/or RAM. If you're not already doing this, make a habit of restricting your searches, like 'title:<blahblah>' or 'authors:'blahblah'. Calibre then only searches that table instead of the entire metadata database, and that can make a significant difference.

Last edited by mbovenka; 10-18-2019 at 07:14 AM.
mbovenka is offline   Reply With Quote
Old 10-18-2019, 09:41 AM   #8
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 31,062
Karma: 60358908
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Have you done Library Maintenance?
That garbage collects the DB. Deadwood (entries) pile up over time and the DB bloats with obsoleted (flagged) entries
theducks is offline   Reply With Quote
Old 10-18-2019, 04:13 PM   #9
Dorian67
Member
Dorian67 began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Oct 2019
Device: Amazon Kindle
....
@mbovenka: "...restricting your searches, like ...." makes the search much faster!
Dorian67 is offline   Reply With Quote
Old 10-18-2019, 04:24 PM   #10
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 31,062
Karma: 60358908
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Quote:
Originally Posted by Dorian67 View Post
....
@mbovenka: "...restricting your searches, like ...." makes the search much faster!
good point.

There is a setting for that so you don't have to remember to be explicit Author: name
Preferences: Searching
Attached Thumbnails
Click image for larger version

Name:	Searching.JPG
Views:	96
Size:	117.3 KB
ID:	174350  
theducks is offline   Reply With Quote
Old 10-19-2019, 06:16 PM   #11
Dorian67
Member
Dorian67 began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Oct 2019
Device: Amazon Kindle
I already set the preferences to author and title.

By the way. I run calibre (64bit) on Windows 10 (64) with 12 GB RAM with an AMD Phenom II X4 905e Prozessor. There are not many other programs running in the background.

The filesize of the metadata.db has 200 MB.

When I start a search the ressourcemonitor shows the following data during the search (see attachment). The searh takes about 3-4 minutes, when I not use "Author:" or "Title".

Dorian
Attached Thumbnails
Click image for larger version

Name:	2019-10-20_00h00_58.png
Views:	75
Size:	17.3 KB
ID:	174372  
Dorian67 is offline   Reply With Quote
Old 10-20-2019, 02:37 AM   #12
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 9,896
Karma: 13806776
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Kindle PaperWhite SE 11th Gen
Quote:
Originally Posted by Dorian67 View Post
I already set the preferences to author and title.
Did you also check the "Limit the searched metadata" box? The search isn't limited to what you enter in the box until you check this option.
Attached Thumbnails
Click image for larger version

Name:	Search Preferences.jpg
Views:	62
Size:	155.1 KB
ID:	174386  

Last edited by DoctorOhh; 10-20-2019 at 02:43 AM.
DoctorOhh is online now   Reply With Quote
Old 10-20-2019, 10:26 AM   #13
Dorian67
Member
Dorian67 began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Oct 2019
Device: Amazon Kindle
@DoctorOhh: "...Did you also check the "Limit the searched metadata" box?..." . Yes it was already set. That reduced the time already, because first I had here also series and tags.

Also I deactivated all the plugins for test. But it had no positive effects.
Dorian67 is offline   Reply With Quote
Reply


Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Batch change metadata of books in library bmcs Calibre 4 10-05-2019 11:32 AM
SQLite DB and books on different drives pweasel Library Management 4 07-24-2016 12:19 AM
Scan hard drives for books not yet in catalog. birkmaggs Library Management 4 02-18-2016 02:00 PM
Split a Library acroos two drives. alanHd Library Management 4 06-03-2013 10:50 AM
Merge Formats library into Metadata library? Or, Add Format in bulk? Sabardeyn Library Management 5 01-23-2013 06:00 AM


All times are GMT -4. The time now is 04:10 PM.


MobileRead.com is a privately owned, operated and funded community.