02-04-2016, 10:48 PM | #1 |
Connoisseur
Posts: 77
Karma: 10
Join Date: Sep 2011
Device: Nook, Boox C67ML
|
Dropbox and Calibre metadata
First, let me preface this by saying I know all about 'one computer at a time' rule. I only have one instance of Calibre, running on Windows 10. (All the other Dropbox access is via read-only mobile apps.) There are no changes happening that are not from that one instance of Calibre.
However, often, when updating metadata, or when adding a new fanfic, I often get an error saying that Calibre could not write metadata. This error goes away if I pause Dropbox. I'm pretty certain this is because Dropbox is currently uploading the .opf file, because Dropbox's 'recently changed' list always has metadata.opf at the top after it happens. I...do not understand what is going on here. Surely Dropbox is only *reading* the file when uploading, and will let others update it. If not, I'd be having problems in all sorts of other apps when saving files. Also, this is some amazing timing to keep happening so often on such small files (Which really should be instantly read and written.), and never happening once with my metadata.db! That's the thing that is really confusing me...these are tiny 20k files, they should be locked by Calibre, instantly written to disk, and then unlocked. And then locked by Dropbox (Or actually not!), instantly pulled into memory, and unlocked, and then uploaded. And while it doesn't happen on *every* metadata change, but but it isn't some once-in-a-blue-moon race condition where all the stars misalign perfectly...I can make it happen by changing the metadata on a dozen books. I saw a mention the other day when trying to figure this out that Calibre does some sort of check to see if someone else is using a file before it writes to it. I think there might be something weird going on there. |
02-04-2016, 11:01 PM | #2 |
Connoisseur
Posts: 77
Karma: 10
Join Date: Sep 2011
Device: Nook, Boox C67ML
|
Actually, the thing I *really* don't understand is how this is causing a Calibre problem.
Even if there was some sort of conflict, Dropbox would have absolutely no reason to have a random file locked *before* it was changed. So what must be happening is that Calibre writes to the file, *pauses*, and then tries to write again but see that Dropbox has the file open for reading. I...can't figure out why Calibre would be doing that on a metadata change. |
Advert | |
|
02-04-2016, 11:02 PM | #3 |
Ex-Helpdesk Junkie
Posts: 19,421
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
|
It is because of file locking.
For what it's worth, although I usually pause Dropbox while calibre is running, I sometimes forget. I have never gotten those errors. I use linux, so the files aren't locked (Dropbox can read them while calibre writes them). Coincidence? I think not... |
02-04-2016, 11:06 PM | #4 |
Bookaholic
Posts: 14,391
Karma: 54969924
Join Date: Oct 2007
Location: Minnesota
Device: iPad Mini 4, AuraHD, iPhone XR +
|
EDIT: Nevermind. I see eschwartz has got you covered.
|
02-05-2016, 11:02 AM | #5 |
Connoisseur
Posts: 77
Karma: 10
Join Date: Sep 2011
Device: Nook, Boox C67ML
|
Yes, I *know* it's because of file locking.
I am asking two things: 1) Why is Calibre opening the file, replacing the contents, closing it (At which point Dropbox steps in and tries to read it) and then Calibre tries to open it *again* a split second later? Why does Calibre not just keep it open while changing it, which would result in Dropbox waiting? Like I said, this isn't happening because I repeatedly change the metadata of a single file, and I'm getting caught because my previous edit hasn't been uploaded yet. I would understand *that* happening, and be recommending solution #2 below. But that *doesn't* happen. It's happening when I edit the metadata of a bunch of books at once. Which means, as far as I can guess, that Calibre is opening and closing metadata.opf from 1-20 (in order?), and dropbox starts reading them...and then Calibre opens *all those files again*, presumably in order again, and thus trips over Dropbox, which had made it to file number 8 or whatever. Why is opening files again?! What is this second pass that trips over anything that is reading recently modified files?! 2) Considering this weird double-update system that Calibre seems to want to use of editing files, if Calibre can't open a file, why can't Calibre just *move that update request to the end up of the queue* instead of throwing an error, and thus try again a second later? Or, heck, write out a metadata.opf.new file and move it over a second later? |
Advert | |
|
02-05-2016, 11:18 AM | #6 |
Connoisseur
Posts: 77
Karma: 10
Join Date: Sep 2011
Device: Nook, Boox C67ML
|
Now I'm wondering if this might be something to do with the fanfic plugin, because that is actually where it seems to happen most often, usually when adding or updating using the plugin.
Perhaps Calibre is changing something, and then the plugin is. |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Calibre Cloud Pro and Metadata.db Sync Dropbox | da_jane | Related Tools | 24 | 02-16-2014 07:19 PM |
Calibre and Dropbox | deannah | Library Management | 1 | 07-30-2013 05:45 PM |
Minor Dropbox Metadata mishap | evans | Related Tools | 1 | 02-17-2012 10:03 PM |
How to use Dropbox with Calibre | BAD18 | Related Tools | 5 | 11-09-2011 08:52 AM |
Using dropbox and Calibre | totaltech | Related Tools | 7 | 04-29-2011 02:28 PM |