Is it safe for two copies of Calibre to access the same database?

Halo · 11-29-2012, 05:31 PM

I have a Linux file server at home that I use for storing the majority of my data. I keep my Calibre library and database there which I access over the network from my Windows laptop. I'd like to set up the Calibre Content Server on my file server so I can access my books via a web browser when I'm away from home.

Is it safe to have both the Calibre Content Server running on my file server and accessing the database while also running another copy of Calibre on my laptop to make changes? Does the content server open the database as read-only? If it is safe, does the content server handle changes to the database automatically or will I have to restart the content server for it to pick up any change? I.e., is the content server caching data?

Also, is it okay if the content server is running an older version of Calibre? Are there database changes happening in recent releases?

Adoby · 11-29-2012, 06:14 PM

No, it is not safe. Calibre is not multi user.

One way to handle it could be to make a backup copy of the library and treat that as a read only version of the library, to be used by the content server. I run a content server like that. I have configured a Virtual Box to automatically copy the latest backup of the Calibre library, using rsync, when starting.

So I have three copies. One main copy that I add books to, and change. One backup. (Actually several backups.) And one copy of the latest backup used by the content server. This means that I can update and backup the main library at any time, and refresh the content server by restarting it.

It is also very, very bad to switch between Windows and Linux. Linux use case sensitive filenames, but Windows don't. Chaos lurking there...

itimpi · 11-29-2012, 06:21 PM

It is safe to have the Content Server accessing the same database as the main Calibre progam as it never changes anything. It is NOT safe to have two copies of the main program accessing the database as they can trample on each others updates.

Halo · 11-29-2012, 07:22 PM

Quote:

Originally Posted by Adoby

One way to handle it could be to make a backup copy of the library and treat that as a read only version of the library, to be used by the content server.

That's a good idea. I'll use that as my fallback if there's no other option.

Quote:

Originally Posted by Adoby

It is also very, very bad to switch between Windows and Linux. Linux use case sensitive filenames, but Windows don't. Chaos lurking there...

I'm okay there as I'm not switching between Windows and Linux. All of my Calibre work is done on Windows. Linux will only be used to run the content server.

afv011 · 11-29-2012, 07:55 PM

Quote:

Originally Posted by Adoby

It is also very, very bad to switch between Windows and Linux. Linux use case sensitive filenames, but Windows don't. Chaos lurking there...

Indeed it is. My library is on Dropbox and I was accessing it from Windows and Linux machines. I got plenty of conflicts due to the case-insensitive nature of Windows, which resulted on ebooks disappearing, the covers disappearing, or both. I have stopped accessing it from Windows and now I rely only on Linux.

kovidgoyal · 01-05-2013, 11:28 PM

The content server is not read only. If for example a calibre upgrade changes the database schema, the content server can cause the contents of metadata.db to change.

The CS can also cause metadata backups to run which alter the db and the file system.

Finally, I plan on adding write features to the content server at some point.

HarryT · 01-06-2013, 03:21 AM

It should be safe enough as long as you can absolutely guarantee that only one person will be accessing it at any one time - eg if you're the only user, and you're accessing it from two different locations. The problems occur if two or more people try to access it at the SAME time.

Halo · 01-31-2013, 01:03 PM

Hi everyone,

I found a solution for this that's working well for me. I'm posting a follow-up so others can try this if they like. It's a bit technical, but if you are using Linux or Mac then this should work for you. So far, I am very happy with the results.

I created a new user on my computer called 'calibre' and installed calibre into /home/calibre/calibre. Then I created a directory called 'library'. In this example my main calibre library is located at /srv/fileshare/Ebooks/Library. I will make a copy of my calibre library to /home/calibre/library which will be used by the calibre content server.

First, make sure the directory for the content server library is empty. You can skip this step the first time you set up your library and content server. Be careful with this command! It removes all files in the directory including all sub-directories. This is one reason that I created a new user called 'calibre' just for the content server. Doing that restricts changes to the 'calibre' user and protects other files on my computer from accidental deletion including the original calibre library.

Code:

rm /home/calibre/library/* -rf

Next, I copy my main calibre library from my file share directory into the newly created library directory that will be used by the content server. I am using the -s option for cp which will create links in the destination that point back to the original files. When I access a file in my content server library, the computer will really read the original file from my main calibre library. This will save disk space on my server since it doesn't have to duplicate every file. The -a option stands for "Archive" and makes sure that the destination creates sub-directories and sets the correct time and date on the destination files (links) among other things.

Code:

cp -as /srv/fileshare/Ebooks/Library/* /home/calibre/library/

Now I remove the links to the database files. We'll make real copies of the files in the next step.

Code:

rm /home/calibre/library/metadata.db
rm /home/calibre/library/metadata_db_prefs_backup.json

We need to copy the actual files for the database rather than create links. This is needed because the content server must be able to write to the database files.

Code:

cp /srv/fileshare/Ebooks/Library/metadata.db /home/calibre/library/
cp /srv/fileshare/Ebooks/Library/metadata_db_prefs_backup.json /home/calibre/library/

The library for the content server is ready. Start up the content server.

Code:

/home/calibre/calibre/calibre-server --with-library /home/calibre/library/ --max-cover=200x266 --daemonize

That's it! Now the content server will serve your books, but it will not interfere with your access of the main library by the main program. If you make changes to the main library, just stop the content server process, run all of the commands in this post from top to bottom, and you will be back in business.

11-29-2012, 05:31 PM	#1
Halo Connoisseur Posts: 96 Karma: 72130 Join Date: Nov 2008 Location: San Francisco Bay Area, CA, USA Device: Sony PRS-505, Kindle DX 1st Gen, Kindle Fire 1st Gen	[SOLVED] Is it safe for two copies of Calibre to access the same database? I have a Linux file server at home that I use for storing the majority of my data. I keep my Calibre library and database there which I access over the network from my Windows laptop. I'd like to set up the Calibre Content Server on my file server so I can access my books via a web browser when I'm away from home. Is it safe to have both the Calibre Content Server running on my file server and accessing the database while also running another copy of Calibre on my laptop to make changes? Does the content server open the database as read-only? If it is safe, does the content server handle changes to the database automatically or will I have to restart the content server for it to pick up any change? I.e., is the content server caching data? Also, is it okay if the content server is running an older version of Calibre? Are there database changes happening in recent releases? Last edited by Halo; 01-31-2013 at 01:04 PM.

11-29-2012, 06:14 PM	#2
Adoby Handy Elephant Posts: 1,736 Karma: 26785668 Join Date: Dec 2009 Location: Southern Sweden, far out in the quiet woods Device: Thinkpad E595, Ubuntu Mate, Huawei Mediapad 5, Bouye Likebook Plus	No, it is not safe. Calibre is not multi user. One way to handle it could be to make a backup copy of the library and treat that as a read only version of the library, to be used by the content server. I run a content server like that. I have configured a Virtual Box to automatically copy the latest backup of the Calibre library, using rsync, when starting. So I have three copies. One main copy that I add books to, and change. One backup. (Actually several backups.) And one copy of the latest backup used by the content server. This means that I can update and backup the main library at any time, and refresh the content server by restarting it. It is also very, very bad to switch between Windows and Linux. Linux use case sensitive filenames, but Windows don't. Chaos lurking there... Last edited by Adoby; 11-29-2012 at 06:35 PM.

01-31-2013, 01:03 PM	#8
Halo Connoisseur Posts: 96 Karma: 72130 Join Date: Nov 2008 Location: San Francisco Bay Area, CA, USA Device: Sony PRS-505, Kindle DX 1st Gen, Kindle Fire 1st Gen	Hi everyone, I found a solution for this that's working well for me. I'm posting a follow-up so others can try this if they like. It's a bit technical, but if you are using Linux or Mac then this should work for you. So far, I am very happy with the results. I created a new user on my computer called 'calibre' and installed calibre into /home/calibre/calibre. Then I created a directory called 'library'. In this example my main calibre library is located at /srv/fileshare/Ebooks/Library. I will make a copy of my calibre library to /home/calibre/library which will be used by the calibre content server. First, make sure the directory for the content server library is empty. You can skip this step the first time you set up your library and content server. Be careful with this command! It removes all files in the directory including all sub-directories. This is one reason that I created a new user called 'calibre' just for the content server. Doing that restricts changes to the 'calibre' user and protects other files on my computer from accidental deletion including the original calibre library. Code: rm /home/calibre/library/* -rf Next, I copy my main calibre library from my file share directory into the newly created library directory that will be used by the content server. I am using the -s option for cp which will create links in the destination that point back to the original files. When I access a file in my content server library, the computer will really read the original file from my main calibre library. This will save disk space on my server since it doesn't have to duplicate every file. The -a option stands for "Archive" and makes sure that the destination creates sub-directories and sets the correct time and date on the destination files (links) among other things. Code: cp -as /srv/fileshare/Ebooks/Library/* /home/calibre/library/ Now I remove the links to the database files. We'll make real copies of the files in the next step. Code: rm /home/calibre/library/metadata.db rm /home/calibre/library/metadata_db_prefs_backup.json We need to copy the actual files for the database rather than create links. This is needed because the content server must be able to write to the database files. Code: cp /srv/fileshare/Ebooks/Library/metadata.db /home/calibre/library/ cp /srv/fileshare/Ebooks/Library/metadata_db_prefs_backup.json /home/calibre/library/ The library for the content server is ready. Start up the content server. Code: /home/calibre/calibre/calibre-server --with-library /home/calibre/library/ --max-cover=200x266 --daemonize That's it! Now the content server will serve your books, but it will not interfere with your access of the main library by the main program. If you make changes to the main library, just stop the content server process, run all of the commands in this post from top to bottom, and you will be back in business.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
sync 2 copies of Calibre?	mark_e_h	Related Tools	3	03-11-2012 09:30 PM
Two copies when fetching news in Calibre	wiltayx	Devices	2	12-04-2011 08:16 AM
Calibre Database cp Kindle Database	mitch13	Library Management	1	05-22-2011 07:33 PM
iLiad Easy and safe SSH access to your iLiad	narve	iRex Developer's Corner	15	11-30-2006 05:14 PM

11-29-2012, 06:21 PM	#3
itimpi Wizard Posts: 4,552 Karma: 950151 Join Date: Nov 2008 Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)	It is safe to have the Content Server accessing the same database as the main Calibre progam as it never changes anything. It is NOT safe to have two copies of the main program accessing the database as they can trample on each others updates.

01-05-2013, 11:28 PM	#6
kovidgoyal creator of calibre Posts: 43,850 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	The content server is not read only. If for example a calibre upgrade changes the database schema, the content server can cause the contents of metadata.db to change. The CS can also cause metadata backups to run which alter the db and the file system. Finally, I plan on adding write features to the content server at some point.

01-06-2013, 03:21 AM	#7
HarryT eBook Enthusiast Posts: 85,544 Karma: 93383043 Join Date: Nov 2006 Location: UK Device: Kindle Oasis 2, iPad Pro 10.5", iPhone 6	It should be safe enough as long as you can absolutely guarantee that only one person will be accessing it at any one time - eg if you're the only user, and you're accessing it from two different locations. The problems occur if two or more people try to access it at the SAME time.