Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Related Tools

Notices

Reply
 
Thread Tools Search this Thread
Old 02-04-2016, 11:48 PM   #1
DavidTC
Enthusiast
DavidTC began at the beginning.
 
Posts: 26
Karma: 10
Join Date: Sep 2011
Device: Nook, Boox C67ML
Dropbox and Calibre metadata

First, let me preface this by saying I know all about 'one computer at a time' rule. I only have one instance of Calibre, running on Windows 10. (All the other Dropbox access is via read-only mobile apps.) There are no changes happening that are not from that one instance of Calibre.

However, often, when updating metadata, or when adding a new fanfic, I often get an error saying that Calibre could not write metadata. This error goes away if I pause Dropbox. I'm pretty certain this is because Dropbox is currently uploading the .opf file, because Dropbox's 'recently changed' list always has metadata.opf at the top after it happens.

I...do not understand what is going on here. Surely Dropbox is only *reading* the file when uploading, and will let others update it. If not, I'd be having problems in all sorts of other apps when saving files.

Also, this is some amazing timing to keep happening so often on such small files (Which really should be instantly read and written.), and never happening once with my metadata.db! That's the thing that is really confusing me...these are tiny 20k files, they should be locked by Calibre, instantly written to disk, and then unlocked. And then locked by Dropbox (Or actually not!), instantly pulled into memory, and unlocked, and then uploaded.

And while it doesn't happen on *every* metadata change, but but it isn't some once-in-a-blue-moon race condition where all the stars misalign perfectly...I can make it happen by changing the metadata on a dozen books.

I saw a mention the other day when trying to figure this out that Calibre does some sort of check to see if someone else is using a file before it writes to it. I think there might be something weird going on there.
DavidTC is offline   Reply With Quote
Old 02-05-2016, 12:01 AM   #2
DavidTC
Enthusiast
DavidTC began at the beginning.
 
Posts: 26
Karma: 10
Join Date: Sep 2011
Device: Nook, Boox C67ML
Actually, the thing I *really* don't understand is how this is causing a Calibre problem.

Even if there was some sort of conflict, Dropbox would have absolutely no reason to have a random file locked *before* it was changed.

So what must be happening is that Calibre writes to the file, *pauses*, and then tries to write again but see that Dropbox has the file open for reading. I...can't figure out why Calibre would be doing that on a metadata change.
DavidTC is offline   Reply With Quote
Old 02-05-2016, 12:02 AM   #3
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 16,890
Karma: 66657949
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
It is because of file locking.



For what it's worth, although I usually pause Dropbox while calibre is running, I sometimes forget.

I have never gotten those errors.
I use linux, so the files aren't locked (Dropbox can read them while calibre writes them).

Coincidence? I think not...
eschwartz is offline   Reply With Quote
Old 02-05-2016, 12:06 AM   #4
AnemicOak
Bookaholic
AnemicOak ought to be getting tired of karma fortunes by now.AnemicOak ought to be getting tired of karma fortunes by now.AnemicOak ought to be getting tired of karma fortunes by now.AnemicOak ought to be getting tired of karma fortunes by now.AnemicOak ought to be getting tired of karma fortunes by now.AnemicOak ought to be getting tired of karma fortunes by now.AnemicOak ought to be getting tired of karma fortunes by now.AnemicOak ought to be getting tired of karma fortunes by now.AnemicOak ought to be getting tired of karma fortunes by now.AnemicOak ought to be getting tired of karma fortunes by now.AnemicOak ought to be getting tired of karma fortunes by now.
 
AnemicOak's Avatar
 
Posts: 11,629
Karma: 38684189
Join Date: Oct 2007
Location: Minnesota
Device: HDX 8.9, AuraHD, Nook HD+, Kindle 2,3,T , Opus, Nexus7, iPhone 6, etc
EDIT: Nevermind. I see eschwartz has got you covered.
AnemicOak is online now   Reply With Quote
Old 02-05-2016, 12:02 PM   #5
DavidTC
Enthusiast
DavidTC began at the beginning.
 
Posts: 26
Karma: 10
Join Date: Sep 2011
Device: Nook, Boox C67ML
Yes, I *know* it's because of file locking.

I am asking two things:

1) Why is Calibre opening the file, replacing the contents, closing it (At which point Dropbox steps in and tries to read it) and then Calibre tries to open it *again* a split second later? Why does Calibre not just keep it open while changing it, which would result in Dropbox waiting?

Like I said, this isn't happening because I repeatedly change the metadata of a single file, and I'm getting caught because my previous edit hasn't been uploaded yet. I would understand *that* happening, and be recommending solution #2 below. But that *doesn't* happen.

It's happening when I edit the metadata of a bunch of books at once. Which means, as far as I can guess, that Calibre is opening and closing metadata.opf from 1-20 (in order?), and dropbox starts reading them...and then Calibre opens *all those files again*, presumably in order again, and thus trips over Dropbox, which had made it to file number 8 or whatever.

Why is opening files again?! What is this second pass that trips over anything that is reading recently modified files?!

2) Considering this weird double-update system that Calibre seems to want to use of editing files, if Calibre can't open a file, why can't Calibre just *move that update request to the end up of the queue* instead of throwing an error, and thus try again a second later?

Or, heck, write out a metadata.opf.new file and move it over a second later?
DavidTC is offline   Reply With Quote
Old 02-05-2016, 12:18 PM   #6
DavidTC
Enthusiast
DavidTC began at the beginning.
 
Posts: 26
Karma: 10
Join Date: Sep 2011
Device: Nook, Boox C67ML
Now I'm wondering if this might be something to do with the fanfic plugin, because that is actually where it seems to happen most often, usually when adding or updating using the plugin.

Perhaps Calibre is changing something, and then the plugin is.
DavidTC is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Calibre Cloud Pro and Metadata.db Sync Dropbox da_jane Related Tools 24 02-16-2014 08:19 PM
Calibre and Dropbox deannah Library Management 1 07-30-2013 06:45 PM
Minor Dropbox Metadata mishap evans Related Tools 1 02-17-2012 11:03 PM
How to use Dropbox with Calibre BAD18 Related Tools 5 11-09-2011 09:52 AM
Using dropbox and Calibre totaltech Related Tools 7 04-29-2011 03:28 PM


All times are GMT -4. The time now is 01:12 PM.


MobileRead.com is a privately owned, operated and funded community.