Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Related Tools

Notices

Reply
 
Thread Tools Search this Thread
Old 05-23-2010, 04:05 AM   #1
q345
Member
q345 began at the beginning.
 
Posts: 23
Karma: 10
Join Date: May 2010
Device: none
Question Keeping book on NAS but metadata.db on local (Ubuntu)

I've installed Calibre latest (0.6.54) version under Ubuntu Lucid (10.04) and it works fine.

The only problem I have is that when I tried to move my e-book directory location from local hard drive to NAS ( which is mounted as -t cifs ) , calibre reported problems. I did a search on the web and found out that other people experienced this problem as well and the reason is the sqlite3 can't work with NAS.

I've tried (instead of moving e-book directory via preferences) to cheat namely,
I manually copied Calibre Library to NAS, then deleted directory Calibre Library from local harddrive and created the symbolic link to NAS directory with the same name. This doesn't work either.

I was wondering if somebody know the way to bypass this, is it possible to keep metadata.db on local hard drive but book themselves on NAS ?
q345 is offline   Reply With Quote
Old 05-23-2010, 05:04 AM   #2
toddos
Guru
toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.
 
toddos's Avatar
 
Posts: 694
Karma: 822675
Join Date: May 2010
Device: Kobo Aura, Nokia Lumia 920 (Freda)
Set an environment variable named CALIBRE_OVERRIDE_DATABASE_PATH to a local path for your metadata.db file, and leave the library location in Preferences set to your NAS share. Calibre looks for that environment variable (and many others) to override default behavior.

I've had my database and library separated for some time now, for similar reasons to yours (I use a WHS rather than a NAS, though I do keep my metadata.db file on the WHS just in a separate location).
toddos is offline   Reply With Quote
 
Enthusiast
Old 05-23-2010, 05:34 AM   #3
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,031
Karma: 777817
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
Calibre allows you to control the placement of all its files by a combination of settings in Environment Variables and command line parameters.

Attached is the batch file I use for this in Windows. It is easy enough to see how it works and should be simple to adapt the syntax to other OS such as Linux or Mac OS. It can also be used to run calibre off a USB stick or other portable media.
Attached Files
File Type: bat CalibreRun (network).bat (3.0 KB, 189 views)
itimpi is offline   Reply With Quote
Old 05-23-2010, 02:17 PM   #4
q345
Member
q345 began at the beginning.
 
Posts: 23
Karma: 10
Join Date: May 2010
Device: none
Thumbs up Thanks lot, works like a charm ( I wish this option will be in Preferences )

Toddos, Itimpi !

Thanks a lot !

Toddos, a single setenv, as you recommended, solved all the problems.

Itimpi, I appreciate you sent me the bat file. I can easily modify it to work as a shell script in Ubuntu (also will be very useful if I need to run calibre under Windows )

I only wish the option of setting separate paths for Library and Metadata were in the Preferences (may be under Advanced), since as I mentioned, quite a few people reported this problem. At the very least, I would put the short note with warning and instructions in FAQ on calibre website and/or manual

Last edited by q345; 05-23-2010 at 09:27 PM.
q345 is offline   Reply With Quote
Old 05-23-2010, 08:20 PM   #5
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 8,783
Karma: 12516053
Join Date: Feb 2009
Location: North Carolina
Device: Nexus 7
Quote:
Originally Posted by q345 View Post
I only wish the option of setting separate paths for Library and Metadata were in the Preferences (may be under Advanced), since as I mentioned, quite a few people reported this problem. As the very least, I would put the short note with warning and instructions in FAQ on calibre website
Open up a enhancement request. That way maybe it will be added some day.
DoctorOhh is offline   Reply With Quote
Old 05-23-2010, 09:25 PM   #6
q345
Member
q345 began at the beginning.
 
Posts: 23
Karma: 10
Join Date: May 2010
Device: none
I will ( "rotating snakes" are cool BTW )

Is it any other way to discover all the environment variables, other then downloading the sources and search for "os.environ.get" string in the source tree ?

I didn't find any document, but the names are mostly self-explanatory

Last edited by q345; 05-23-2010 at 09:28 PM.
q345 is offline   Reply With Quote
Old 05-24-2010, 12:53 AM   #7
q345
Member
q345 began at the beginning.
 
Posts: 23
Karma: 10
Join Date: May 2010
Device: none
Enhancement ticket #5600 has just been opened

http://bugs.calibre-ebook.com/ticket/5600

Enable separate directories for metadate.db and ebooks library in Preferences Menu
q345 is offline   Reply With Quote
Old 05-24-2010, 05:48 PM   #8
q345
Member
q345 began at the beginning.
 
Posts: 23
Karma: 10
Join Date: May 2010
Device: none
Update : #5600 closed - resolution: wont' fix

The ticket has been closed with explanation that this is not a recommended procedure, so it will not be exposed in Preferences menu.

May be there is an additional headache to support or there are some side effects to it, who knows.

Anyway, a single setenv in .gnomerc has enabled it and it works fine ever since
q345 is offline   Reply With Quote
Old 05-25-2010, 02:42 AM   #9
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,031
Karma: 777817
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
Quote:
Originally Posted by q345 View Post
The ticket has been closed with explanation that this is not a recommended procedure, so it will not be exposed in Preferences menu.

May be there is an additional headache to support or there are some side effects to it, who knows.
I have personally been running in this mode for many months and have not encountered any downsides. My suspicion is that for users who do not know much about how their computers run they could get into all sorts of trouble trying to use this facility with their databases and book store getting out of sync. That is probably why there is reluctance to expose the options in the GUI and make it easy for naive users to find them.

I would like to see the full list of environment variables and command line options used by calibre in this area added to the documentation for future reference.
itimpi is offline   Reply With Quote
Old 05-25-2010, 04:16 AM   #10
q345
Member
q345 began at the beginning.
 
Posts: 23
Karma: 10
Join Date: May 2010
Device: none
advanced features

If it's only a safeguard against naive users, tt can always be added to advanced features with proper warning/disclaimer BTW, even if it's in a normal mode and an uneducated user starts playing with Calibre Library directory, he/she easily can get into all kind of troubles, even without NAS, so I wonder if it's the only reason.

I wish the ticket would be given a little bit more explanation, why it's not recommended ( it was just a single sentence ), for instance is there any negative side effects on the long run, but after all it's a free software and we all should thank the developer since he is clearly doing us a huge favor by developing and supporting it.

Regarding environmental variables in general, may be it's too much to ask. I grepped source directory for os.environ.get method and there is not a lot of them, but writing a manual, especially describing the combinations of them might take too much time. I thought this particular option is reasonable to expose, since NASes are mainstream now and installation is so easy that even "dumb" users now purchase them but it's my private opinion of course
q345 is offline   Reply With Quote
Old 05-25-2010, 04:34 AM   #11
toddos
Guru
toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.toddos ought to be getting tired of karma fortunes by now.
 
toddos's Avatar
 
Posts: 694
Karma: 822675
Join Date: May 2010
Device: Kobo Aura, Nokia Lumia 920 (Freda)
Quote:
Originally Posted by q345 View Post
I wish the ticket would be given a little bit more explanation, why it's not recommended ( it was just a single sentence ), for instance is there any negative side effects on the long run, but after all it's a free software and we all should thank the developer since he is clearly doing us a huge favor by developing and supporting it.
I've been running Calibre that way for more than a month now with no ill effects. That doesn't mean there aren't any, just that I haven't run into them yet.


Quote:
Originally Posted by q345 View Post
Regarding environmental variables in general, may be it's too much to ask. I grepped source directory for os.environ.get method and there is not a lot of them, but writing a manual, especially describing the combinations of them might take too much time.
So contribute to the project by writing a manual?
toddos is offline   Reply With Quote
Old 05-25-2010, 11:34 AM   #12
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,453
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
The reason is simple, this kind of customization is fragile. The calibre codebase is not designed to support it and things can break. If you are careful and lucky enough, they wont break, but this is very much a you broke the warranty kind of deal.

Furthermore, I am not willing to guarantee that this technique will always be available in the future. Which becomes implied if I add it to the preferences.
kovidgoyal is offline   Reply With Quote
Old 05-25-2010, 12:15 PM   #13
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,293
Karma: 5495472
Join Date: Aug 2009
Location: The (original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by itimpi View Post
I have personally been running in this mode for many months and have not encountered any downsides. My suspicion is that for users who do not know much about how their computers run they could get into all sorts of trouble trying to use this facility with their databases and book store getting out of sync. That is probably why there is reluctance to expose the options in the GUI and make it easy for naive users to find them.

I would like to see the full list of environment variables and command line options used by calibre in this area added to the documentation for future reference.
Desktop databases are fragile by their nature. Many processes happen asynchronously.

Glitch that and things get broken.

Calibre appears to be a hybrid of the O/S file system (author and title folders) and a database to keep track of said contents (along with other stuff). Queuing files to get written to a networked device and having a local (faster), database is asking for one to get updated without the other completing

Suppose the mount point gets confused?
Kovid kept the Metadata DB in the same path as the books for a good reason
They are a mated pair.
It is your foot to shoot off. :/
You can store Gasoline near a water heater, without it ALWAYS catching fire. . . That does not make it a good idea.
theducks is offline   Reply With Quote
Old 05-25-2010, 12:31 PM   #14
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,031
Karma: 777817
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
Actually if you are going to run off a NAS it is probably MORE reliable to have the database locally. File level operations (open/close/rename/copy etc) across the network tend to be extremely reliable, whereas SQLite database updates are fragile and low performance.

The biggest issue with having the database locally is likely to be user error when using Calibre on multiple machines with not always working off the latest metadata database. I reduce the likelihood of that happening by having an automatic sync of the metadata database back to the NAS whenever Calibre is closed down. I am also the only person updating my Calibre library so am not subject to someone attempting to make changes asynchronously to me.
itimpi is offline   Reply With Quote
Old 05-25-2010, 01:28 PM   #15
q345
Member
q345 began at the beginning.
 
Posts: 23
Karma: 10
Join Date: May 2010
Device: none
Reliability

There is a digital music player box called Squeezebox from Logitech which I have had for 5 years. The server maintains database of one's digital music (they used sqlite and also mysql). I have never had any problem for 5 years having my mp3 files on NAS. Tens of thousend of peoples maintains they collections on NAses while having database file localy, I don't recall somebody complained. ( It's an official option not a hack )

I don't know if it's a factor, but in case of Logitech Squeezebox, files are not physically get copied, which is a major differences, of course. Many other similar devices applications like amarok music player on Ubuntu, Media Monkey on Windows uses the same approach. They all have an option 'rebuild database' which scan the directory and rebuild database and those applications are by no means intended for geeks.

May be Calibre should have a similar option - rebuild database, which scan Calibre Library directory and generate metadata.db as a result. Cost of database inconsistency will be just few minutes.

Last edited by q345; 05-25-2010 at 01:33 PM.
q345 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Calibre hangs on startup after moving to NAS smartin Calibre 4 09-26-2010 04:21 PM
Metadata and book covers Magic Man Calibre 3 09-05-2010 06:20 AM
Seriously thoughtful Anyone have any experience of NAS devices? HarryT Lounge 9 12-10-2009 04:50 AM
Library on NAS server? bous Calibre 5 11-26-2009 05:03 AM
software for book keeping ? mgph Sony Reader 8 06-06-2009 09:55 PM


All times are GMT -4. The time now is 02:37 AM.


MobileRead.com is a privately owned, operated and funded community.