Calibre + Dropbox = unclouded happiness

[dpierron]

NEW The tool is now called calibre2opds and it has its own website, and thread here on MobileRead...

NEW I programmed a tool based on Calibre2Web that can be used on all platforms (Windows, Macs and Unixes) to generate the Stanza catalogs.

It is hosted on Sourceforge and I intend on updating it often, so please join the project and make requests !

-------------------------------------------

I'm pretty sure a lot of people here know Dropbox, the (free at first) file sharing and synchronization service...

I use it to sync the files on my two Macs, and my work PC ; it's a very easy to use and efficient service.

Recently, I began syncing my Calibre library, so the books I edit/add/remove on either machine would be available on all the others ; Calibre is a multi-platform marvel, so this syncing makes a lot of sense.

And I thought of something : if I generate the stanza.xml file from the Calibre database (using the calibre-db command), maybe it could be automatically accessible from my iPhone, now that the whole Calibre library is synchronized and available on my Dropbox account ?

Guess what ? It works ! Unfortunately, it's still a manual process, but I'm pretty sure it could be possible for our generous Kovid to change it into something automatic...

Here are the steps needed to make this happen :

1. Publish your Calibre library in a directory of the Public part in your Dropbox account
   
   The library must be in a public part of your Dropbox account, because you will need the URL to the stanza.xml file in the last step. If you do not want to make it public, you will still be able to browse your (private) dropbox account website from inside Stanza, but it's another story...
   
   To get the URL to your Calibre library, simply right-click on a file in the folder and select "Copy public link" from the "Dropbox" menu ; let's call this URLTOSHAREDCALIBRELIBRARY : mine is http://dl.dropbox.com/u/788541/calib...ary/stanza.xml (fake number, of course).
   
   
2. export the Calibre database into a stanza.xml file
   We need to use the calibredb program, located in the Calibre program folder ; find out the path to this file ; let's call it PATHTOCALIBREDB (use french quotes around it if there are spaces in it) : mine is "c:\Program Files\Calibre2\calibredb.exe".
   
   Then, we need the path to the Calibre library folder ; let's call it CALIBRELIBRARYFOLDER (again, use french quotes around it if there are spaces in it) : mine is "c:\data\dropbox\My Dropbox\Public\calibre_library"
   
   Here is the command used to generate the stanza.xml file :
   
   PATHTOCALIBREDB list --output-format stanza --prefix URLTOSHAREDCALIBRELIBRARY > CALIBRELIBRARYFOLDER\stanza.xml
   
   (replace the variables with the ones we discussed earlier)
   
   As an example, here is the command line I am using :
   
   "c:\Program Files\Calibre2\calibredb.exe" list --output-format stanza --prefix http://dl.getdropbox.com/u/788541/calibre_library > "c:\data\dropbox\My Dropbox\Public\calibre_library\stanza.xml"
   
   
3. correct the stanza.xml file
   
   Some characters are encoded in a way that make Stanza frown : simply replace them using a plain text editor ; replace '%28' with '(' and '%29' with ')' (skip the quotes, obviously).
   
   
4. configure your Stanza program to make it access your new online library
   
   In Stanza (on your iPhone for example), create a new online stanza catalog and enter the URL to your stanza.xml file ; this URL is URLTOSHAREDCALIBRELIBRARY/stanza.xml : mine is http://dl.dropbox.com/u/788541/calib...ary/stanza.xml.

That's all !

Of course, if the stanza.xml file could be generated automatically with every modification of the database, it would be even better, but even now it's very convenient...

Thanks again Kovid for Calibre, the Lexycle people for Stanza and Dropbox for being free...

PS : if I made a mistake in my explanation, or if something was not clear enough, ask questions and I'll be happy to try and explain better...

[kjk]

So...you are accessing your entire Calibre library on Stanza iPhone from anywhere? Over 3G? That...is pretty awesome.

[dpierron]

Over Wifi, 3G, even over Edge : the file sizes are usually small enough to make it possible even with a very low bandwidth.

[Gideon]

That's bloody brilliant

[dpierron]

Thanks for the bloody compliment

Unfortunately, Kovid seems to disagree with my opinion that it would be better if Calibre itself triggered the generation of the stanza.xml catalog ; his point of view is that it would be easy to do with a cron job, and I'm trying to demonstrate that not everyone (nor every operating system) is capable of managing cron jobs... But I'm afraid it's a lost cause
Anyhow, even if it has to be done manually, it's still great and it's still thanks to Kovid and his amazing work that we're able to do it !

[dpierron]

And guess what is already corrected in today's Calibre release (0.6.24) ?

calibredb list: Do not URL encode the href fields for the stanza output format

Thanks Kovid, that's one small step for you, and a giant's leap for us, Dropbox users

[rcuadro]

I finally got it working This is awesome! It gives me almost the same relationship between my iPhone and Stanza as I have with my Kindle and Amazon

[dpierron]

Yes, I agree ! I'm having the same feeling.

Let's not upgrade the calibredb tool to version 0.6.24, because the change Kovid made for Stanza catalog export is causing a problem : the spaces are not converted into %20 characters, and therefore the stanza.xml file is not readable by Stanza.

Better keep 0.6.23 until it is fixed, even if this means that we must manually convert '%28' and '%29' into '(' and ')' characters ...

[itimpi]

Imust admit I did not understand why Kovid made that change. If a URL is not correctly escaped it does not get handled correctly by Stanza. I guess there must have been a reason otherwise he would not have done it?

Has a ticket been raised to point out this is causing problems?

[dpierron]

In fact it's all my fault : I asked Kovid (in this ticket) to change the behavior of calibredb so that we would not have to manually convert %28 into '(' and %29 into ')'

And then the change he made disabled urlencoding of the links in the stanza.xml file, which is overkill (but I should have said it beforehand).

He couldn't test it, because he would need an iPhone with Stanza...

[rcuadro]

I have been using .6.24 since it was released and it works well enough for me for the time being... especially since I am in the middle of a book on my Kindle
The best thing is that: 1) I now have the ability to have remote access to all my non Amazon books. 2) I have a central location for my eBooks

It would have been even better if I could have been able to access Dropbox from my Kindle the same way as Stanza on the iPhone.

[rcuadro]

You know... you are right...if there was a way to get Calibre to automatically generate the stanza.xml automatically when changes are made it would be awesome

[dpierron]

Originally Posted by rcuadro

I have been using .6.24 since it was released and it works well enough for me for the time being...

Maybe it's because none of your file paths contain spaces ?

[tlrowley]

Originally Posted by rcuadro

It would have been even better if I could have been able to access Dropbox from my Kindle the same way as Stanza on the iPhone.

Forgive me you already know this, but the newer versions of the Calibre server detect the mobile browser of the Kindle. I'm able to download all my non-Amazon books (I also store the Amazon ones, too), directly to my Kindle. Searching/sorting all works - it's fabulous!!

It's not Dropbox, since you have to keep your own computer running, but it works great. I do use Dropbox, too - I keep my library on my laptop synced (via Dropbox) to the iMac that we use as our server.

[mores]

What are "cron jobs"?





Hope that helps your cause!