Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 04-20-2017, 01:25 PM   #1
paperrhino
Junior Member
paperrhino began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Apr 2017
Device: Android
Headless server with non-default DB location

I'm experimenting with setting up a headless calibre-server on Ubuntu Server 17.4. I've come a long way but am facing a bit of a problem.

I've got it running as a service but my books are on a file share so I've set the CALIBRE_OVERRIDE_DATABASE_PATH=/home/calibre/metadata.db environment variable so it saves the DB not on the CIFS mounted file system per the warnings in the FAQ.

I've even managed to install and configure one plugin from the command line. so far so good.

My problem is with adding books. When I try to run the following command (with the override DB environment variable correctly set) calibre creates a new metadata.db in the library folder rather than editing the metadata.db that calibre-server is actually using.

Code:
xvfb-run calibredb add /mnt/calibre/toadd/ -r --with-library /mnt/calibre/library
When I list the contents of the newly created /mnt/calibre/library/metadata.db I see only the new books. When I list the contents of the "real" metadata.db I get all the old books and none of the new ones.

Is alternative DB locations not supported by calibredb or am I doing something wrong?

Many thanks.
paperrhino is offline   Reply With Quote
Advert
Old 04-20-2017, 05:38 PM   #2
gbm
Wizard
gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.
 
Posts: 1,163
Karma: 3620842
Join Date: Jun 2010
Device: Windows 10 tablet,Hisence Sero 7 Pro RIP, Nook STR, jetbook lite
Quote:
Originally Posted by paperrhino View Post
I'm experimenting with setting up a headless calibre-server on Ubuntu Server 17.4. I've come a long way but am facing a bit of a problem.

I've got it running as a service but my books are on a file share so I've set the CALIBRE_OVERRIDE_DATABASE_PATH=/home/calibre/metadata.db environment variable so it saves the DB not on the CIFS mounted file system per the warnings in the FAQ.

I've even managed to install and configure one plugin from the command line. so far so good.

My problem is with adding books. When I try to run the following command (with the override DB environment variable correctly set) calibre creates a new metadata.db in the library folder rather than editing the metadata.db that calibre-server is actually using.

Code:
xvfb-run calibredb add /mnt/calibre/toadd/ -r --with-library /mnt/calibre/library
When I list the contents of the newly created /mnt/calibre/library/metadata.db I see only the new books. When I list the contents of the "real" metadata.db I get all the old books and none of the new ones.

Is alternative DB locations not supported by calibredb or am I doing something wrong?

Many thanks.
I think you need the share UUID ex:
Code:
/mnt/1589cd3b-4281-4652-af95-1aed1a229bcc
Look in /opt/calibre/resources/ at the contents of calibre-portable.sh.

Edit


Verily the path and that they are mounted of your share using a terminal and the cd and l commands.

The only device I have that does not mount with a UUID is my NOOK.
bernie

Last edited by gbm; 04-20-2017 at 05:54 PM.
gbm is online now   Reply With Quote
Old 04-21-2017, 09:57 AM   #3
paperrhino
Junior Member
paperrhino began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Apr 2017
Device: Android
Thank you for your reply. I'm not certain I fully understand what you are asking me to do.

I've looked at calibre-portable.sh and I see where it creates a conf file and in there I see where it sets the CALIBRE_OVERRIDE_DATABASE_PATH which looks a lot like what I'm already doing.

My share is a cifs mounted share so I don't think it has a uuid beyond the //machine/dir path.

And I don't understand how that would help. I wanted to have my books stored in one directory and my metadata.db stored in another. The problem wasn't that calibredb couldn't find the books library folder, it was that it was ignoring the CALIBRE_OVERRIDE_DATABASE_PATH environment variable.

In any case, I determined the big problem with having the library on a network share is trying to run multiple instances of Calibre on the same library. Since I'm only ever going to have this one server putting the library on a network share has minimal risk of corruption for me. So I'm just using the default location for metadata.db for now and it is working just fine.

Now, as I purchase eBooks I can drop them into a toadd folder and a cron job will automatically add them to the Calibre library. Then I can browse and download books through the web or apps like Calibre Companion.

Thanks again for the help.
paperrhino is offline   Reply With Quote
Old 04-21-2017, 10:47 AM   #4
gbm
Wizard
gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.gbm ought to be getting tired of karma fortunes by now.
 
Posts: 1,163
Karma: 3620842
Join Date: Jun 2010
Device: Windows 10 tablet,Hisence Sero 7 Pro RIP, Nook STR, jetbook lite
Looks to me like you could use calibre-portable.sh to do what you need in setting the variables and changing the line"
Code:
$CALIBRE --with-library "${CALIBRE_LIBRARY_DIRECTORY}"
to
Code:
$CALIBRE-SERVER --with-library "${CALIBRE_LIBRARY_DIRECTORY}"
You have it working that all that matters.
bernie
gbm is online now   Reply With Quote
Old 04-24-2017, 10:24 AM   #5
paperrhino
Junior Member
paperrhino began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Apr 2017
Device: Android
I did review the script and I am doing the exact same thing as the script. I have the CALIBRE_OVERRIDE_DATABASE_PATH environment variable set just as it does in that script and I pass the path to the books using --with-library.

My metadata.db file is not located in the default location, hence my setting of the override environment variable.

calibre-server runs just fine. It uses the metatadata.db file I indicate in the override environment variable and it puts the book files in the directory I indicate in the --with-library flag.

The problem is calibredb seems to ignore the override environment variable. Therefore I cannot add books to my library from the command line when metadata.db is not in the default location.

Thanks for trying to help. I just wanted to make is clear what the problem was in case someone else has the same problem in the future.
paperrhino is offline   Reply With Quote
Advert
Old 04-28-2017, 12:46 PM   #6
paperrhino
Junior Member
paperrhino began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Apr 2017
Device: Android
Quote:
Originally Posted by gbm View Post
Looks to me like you could use calibre-portable.sh to do what you need in setting the variables and changing the line"
Code:
$CALIBRE --with-library "${CALIBRE_LIBRARY_DIRECTORY}"
to
Code:
$CALIBRE-SERVER --with-library "${CALIBRE_LIBRARY_DIRECTORY}"
You have it working that all that matters.
bernie

But I already am seeing the environment variables like in the Shell script. And I'm calling calibre-server just like in the Shell script. All of that works.

The problem is when I call the separate program calibredb, this program ignores the environment variables. And since the portable script doesn't mention calibredb at all it is no help.

While I do have it working, it would be nice to know if calibredb is capable of supporting a non-default location for the db file. If calibredb, and calibre-server support this it seems to reason that calibredb should as well.
paperrhino is offline   Reply With Quote
Old 04-28-2017, 06:56 PM   #7
Krazykiwi
Zealot
Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.
 
Posts: 123
Karma: 2153672
Join Date: Jan 2013
Device: Too many random androids to list
This is probably stating the obvious, but it's ridiculously easy to set a bunch of env vars, and then put yourself outside the context you've set them in. Like logging in as another user who doesn't have it set in the first place, su'ing without the -, or forgetting to export the vars in a script.

Since it's headless, you're remote logging in, and I assume not directly as the user the service is running as.

Does wrapping calibredb in say, a calibredb-wrapper script that does nothing but set and export the vars you need, and runs calibredb work?
Krazykiwi is offline   Reply With Quote
Old 05-05-2017, 12:15 PM   #8
paperrhino
Junior Member
paperrhino began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Apr 2017
Device: Android
Quote:
This is probably stating the obvious, but it's ridiculously easy to set a bunch of env vars, and then put yourself outside the context you've set them in. Like logging in as another user who doesn't have it set in the first place, su'ing without the -, or forgetting to export the vars in a script.
Indeed but as part of my debugging I've echoed the environment variable immediately before running calibredb to verify that the variable is set.

Quote:
Since it's headless, you're remote logging in, and I assume not directly as the user the service is running as.
That is correct, however, I have also tried it running as the calibre user using:

sudo -u calibre
su - calibre
sudo -u calibre /bin/bash

In each case I've verified that the environment variable is set before running calibredb.

Quote:
Does wrapping calibredb in say, a calibredb-wrapper script that does nothing but set and export the vars you need, and runs calibredb work?
I've done this as well.

In every case, rather than using the metadada.db file that is in /home/calibre and defined in the environment variable it creates a new one in /mnt/books/library.

Has anyone else actually done this? It seems unlikely that I'm the first person to try. Either calibredb doesn't pay attention to the environment variable or I'm doing something stupid.
paperrhino is offline   Reply With Quote
Old 05-06-2017, 05:30 PM   #9
Krazykiwi
Zealot
Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.
 
Posts: 123
Karma: 2153672
Join Date: Jan 2013
Device: Too many random androids to list
Quote:
Originally Posted by paperrhino View Post
Indeed but as part of my debugging I've echoed the environment variable immediately before running calibredb to verify that the variable is set.
I figured, but it's always worth asking even the obvious questions when troubleshooting. And at least that's ruled out entirely.

This isn't something I've done before, but I'm watching with interest because it's something I had been planning to do at some point and just hadn't got around to.

Based on your description, and the debugging you've already done, I'm inclined to agree with you: calibredb is not taking any notice of the environment variables. I'd file a bug at this point.
Krazykiwi is offline   Reply With Quote
Old 05-06-2017, 10:26 PM   #10
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 30,853
Karma: 7728435
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I fond that highly unlikely. The code that uses CALIBRE_OVERRIDE_DATABASE is in the database backend. All calibre utilities use that same code.

I'm guessing you only set, not exported the env var. Or xvfb-run is stripping the the env var. Incidentally why are you using xvfb-run? calibre does not need it, it uses a headless QPA backend, which was introduced several years ago, IIRC.
kovidgoyal is offline   Reply With Quote
Old 05-12-2017, 12:09 PM   #11
paperrhino
Junior Member
paperrhino began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Apr 2017
Device: Android
Quote:
Originally Posted by kovidgoyal View Post
I fond that highly unlikely. The code that uses CALIBRE_OVERRIDE_DATABASE is in the database backend. All calibre utilities use that same code.

I'm guessing you only set, not exported the env var. Or xvfb-run is stripping the the env var. Incidentally why are you using xvfb-run? calibre does not need it, it uses a headless QPA backend, which was introduced several years ago, IIRC.
The only tutorial I found that showed how to set up a headless server that worked for me used it. When I tried to run calibredb without using it I would get errors.
paperrhino is offline   Reply With Quote
Old 05-12-2017, 12:46 PM   #12
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 30,853
Karma: 7728435
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Really, what errors?
kovidgoyal is offline   Reply With Quote
Reply

Tags
metadata.db calibredb

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Conversions via headless calibre server? mrdrew Conversion 1 09-07-2016 10:40 PM
Content server on ubuntu 12.04 headless server ImageMagick: libltdl.so.7 Cool Javelin Calibre 5 04-08-2015 02:20 AM
DEBIAN Headless SERVER sMystero Devices 10 04-03-2015 06:55 PM
Headless Calibre Server greene48 Library Management 17 10-28-2013 01:09 AM
Calibre Headless Server dearleuk Library Management 2 12-02-2012 04:04 AM


All times are GMT -4. The time now is 10:04 AM.


MobileRead.com is a privately owned, operated and funded community.