Two installations using the same bookstore

[chaley]

Originally Posted by victor50

I just added two books, converted them to EPUB and edited metadata, while on the other PC Calibre was open (and in server-mode). Database functions perfectly. The only thing is that it didn't refresh and there wasn't a possibility to force a refresh. After restart it did show the added books ok. Then, I think in the helpfile topic "CALIBRE_OVERRIDE_DATABASE_PATH" it says this is usefull when the external drive doesn't support locking, meaning that multiuser use is possible. Maybe @kovidgoyal could shed some more light on the matter

FWIW I have been working with Kovid on calibre for more than 10 years, including in the database layer. I have some idea of what I am talking about.

Calibre loads the DB into memory when it starts. If you make changes it starts with the memory copy, changes it, then writes it to the DB. It does not check if the DB had been changed. If another calibre instance is running then the two memory copies are now out of sync. Changes in the other instance will overwrite those made by the first instance. For a certain class of changes the result will be a database that is not consistent with the rest of the library, sometimes to the point where there is significant data loss.

You can post all the "tests" you want. You can believe me or not. Neither changes the underlying fact that running multiple instances of calibre of the same library will eventually destroy your library.

[DNSB]

Originally Posted by victor50

I didn't argue with the creator AFAIK. And posting facts can't be a problem I hope? Especially when someone says the database will be destroyed, when it isn't.

Perhaps you might want to take a look at the number of threads in these forums where someone goes "I've been running calibre with the library on a NAS, on Dropbox, on Google Drive, etc. and it worked for years but now my library has errors.

As for not arguing with the creator? You might want to check the calibre FAQs. Look for [URL="https://manual.calibre-ebook.com/faq.html#i-am-getting-errors-with-my-calibre-library-on-a-networked-drive-nas"I am getting errors with my calibre library on a networked drive/NAS?[/URL] for his stance on storing your calibre library on a networked drive.

[victor50]

The problem I started with had little to do with all this and I have been (single-)using Calibre for years from my NAS, indeed storing the database locally. Don't know why anymore, anyway I forgot about it in a way. For the (good?) reason mentioned above I wanted to be able to access the books by way of Calibre from two PC's. Now Chalye tells me the database will be destroyed which doesn't seem correct from my experience. It can become inconsistent as he describes in a later post. As I am still "single-using" chances are small if non-existent. I am willing to take that small risk, probably resulting in deleting and adding a book again. Then I make backups regularly. There seems to be a possibility to run a Calibre server on my Synology NAS. I guess it would be truly multi-user. I would prefer that, but the installation looked too much for my limited Linux knowledge.

[theducks]

'Server' is a single connection to the DB
The 'Server' does the user request management.

[DNSB]

Originally Posted by victor50

Now I can use Calibre on both PC's, even at the same time.

Originally Posted by victor50

As I am still "single-using" chances are small if non-existent. I am willing to take that small risk, probably resulting in deleting and adding a book again.

It's hard to reconcile your two statements where you state you are accessing calibre on both PCs simultaneously and that you are "single-using".

And the result of database inconsistency can be far more serious than "probably resulting in deleting and adding a book again."

At one point while experimenting, I managed to render a calibre database unusable after adding and deleting books on three machines, vacuuming the database and then closing all instances as close to the same time as I could. I did recover most of the library by deleting the metadata.db and then rebuilding the database and ignoring the errors from having duplicate IDs.

Your computer, your way of using it. Make sure you keep your backups up to date.

[ownedbycats]

Originally Posted by chaley

FWIW I have been working with Kovid on calibre for more than 10 years, including in the database layer. I have some idea of what I am talking about.

Calibre loads the DB into memory when it starts. If you make changes it starts with the memory copy, changes it, then writes it to the DB. It does not check if the DB had been changed. If another calibre instance is running then the two memory copies are now out of sync. Changes in the other instance will overwrite those made by the first instance. For a certain class of changes the result will be a database that is not consistent with the rest of the library, sometimes to the point where there is significant data loss.

You can post all the "tests" you want. You can believe me or not. Neither changes the underlying fact that running multiple instances of calibre of the same library will eventually destroy your library.

Also, sqlite is not a server. It's not designed to be used in such a manner. If you want multiple users, use the content server.

[Quoth]

Originally Posted by ownedbycats

Also, sqlite is not a server. It's not designed to be used in such a manner. If you want multiple users, use the content server.

Yes. The sqlite is just a sort of api. It's simply accessing files.
Ultimately a server based database is just accessing files, but the client is only accessing a server process and the files are private to it. The Server process will track row and table usage and lock rows or sometimes tables or indexes.

Even then, for success, the Client app has to be written multuser aware and do something sensible when it finds it can't recreate a table or update a row or rebuild an index.