Catalog oddities

[tamhas]

I decided to update Calibre with some new titles in my Kindle today and have found some peculiarities. There were about 10 titles showing in the Kindle as not being in the library. This itself is not unusual, but the first two were books my wife had sent me by e-mail and thus would have been in Calibre prior to being loaded in the Kindle. I added them anyway and converted them to epub (I keep both formats for all books) and all seemed fine except that I had to correct the name from first last to last, first (R.E. Donald). The next book is one that I knew had started downloaded to the Kindle, but I also knew that I had loaded it to Calibre previously because I converted it to epub prior to a recent trip my wife took. I added it to the library and when trying to change the name from first last to last, first it said it couldn't do that because the file was busy.

Looking on the disk I see that I have both a Koryta, Michael folder with two books in it and a Michael Koryta folder which has only one book, the one I just loaded. I have the same situation with the R.E. Donald except there is only one book in the first last version. This latter could simply mean that I failed to fix this when I loaded on 6/29. Perhaps this means that I should have tried matching the library from the Kindle entry first ... and I will try that going forward, but now I have this mess to clean up.

I ran the Library Maintenance Check (had a hard time finding it until I unplugged the Kindle) and I have extra titles for the Koryta and one of the other author's books, an Extra Author for the first last of R.E. Donald. So, I deleted the unwanted first last folders, but now when I go to the Koryta, Michael entry in Calibre for the book I just reloaded, it shows only AZW3 in the formats on the right panel, even though there are both in the directory and if I try a convert to epub, it tells me the book has no ebooks so there is nothing to convert. I suppose I can clean this up by deleting that book from Calibre, reloading from the Kindle, and reconverting, but is there a more direct repair? The R.E. Donald books look OK.

[eschwartz]

It sounds like the folders got unlinked/badlinked from the records somehow -- try the option Restore Database, part of Library Maintenance, to rebuild the database from the book files.

This problem is not unknown when switching libraries between OS's, or when suffering sync conflicts while keeping calibre's library in a cloud storage folder.

[tamhas]

So, a Dropbox burp?

[eschwartz]

Originally Posted by tamhas

So, a Dropbox burp?

VERY likely.

If you sync calibre library to dropbox, the recommended way is pause Dropbox while calibre is running, then do a full sync after calibre is closed.

Otherwise, Dragons (or gremlins) may set fire to your Calibre Library, which would be bad.

In fact, some people go so far as to use filesync tools like rsync, FreeFileSync, DeltaCopy, or Microsoft's SyncToy, to mirror their Calibre library using one-way sync into their Dropbox. So the main library is always canonical, and it is periodically backed up to the Dropbox folder.

[tamhas]

Curiously, I have been running this way for a couple of years and this is the first time I noticed an issue. Manual sync is not appealing since the whole point is to automatically have the catalog available on other machines on demand. And, I tend to open Calibre on my primary machine and leave it running ...

[eschwartz]

Originally Posted by tamhas

Curiously, I have been running this way for a couple of years and this is the first time I noticed an issue. Manual sync is not appealing since the whole point is to automatically have the catalog available on other machines on demand. And, I tend to open Calibre on my primary machine and leave it running ...

Well, it isn't a good idea to open the same file(s) in two programs at the same time.

If you want "automatic" you should script it rather than rely on programs that run automatically.

I'd suggest launching calibre via a script which launches calibre and then waits for it to close before backing up the library to your dropbox folder. You can use rsync for linux, or FreeFileSync for Windows.

In addition to speedily backing up locally (freeing up the main folder to reopen calibre) this ensures that Dropbox cannot accidentally clobber your files via edit clashes -- your primary computer thus maintains the integrity of the canonical library.

I wouldn't recommend leaving calibre always running. The only good reason is to run the server, which is better done anyway through calibre-server(.exe). It will add extraneous load to your computer, and calibre doesn't take all that long to start IME.

[tamhas]

There is no simultaneous use. In fact, if I access from my laptop, my desktop is almost certainly asleep at the time.

[eschwartz]

Originally Posted by tamhas

There is no simultaneous use. In fact, if I access from my laptop, my desktop is almost certainly asleep at the time.

I was talking about having it open on the same computer in both calibre and dropbox. Both are willing to write to the files there, and lock them if necessary -- freezing an extra copy when calibre moves a file/folder but Dropbox still has it open to sync.