Calibredb with DropBox?

[jdege]

I've been keeping my ebooks in Calibre for many years, running it on a series of Linux boxes.

I've always run calibre.service and configured ports through my firewall so I could connect Calibre Companion to my Calibre repository from wherever I might be.

On my Linux desktop I've configured my menus to run scripts that shutdown calibre.service, run the Calibre UI, then start calibre.service back up.

I've just finished moving my KeePass database to DropBox, and I'm thinking about whether to move my Calibre repository to Dropbox, as well.

There's one critical function I've not been about to do from Calibre Companion, and that's to add new books from my mobile device. What I've been doing, instead, is to use sftp to copy the file from my mobile to my Linux server, to ssh onto my Linux server, then using calibredb to add the book to my library. (I have a script that stops and restarts Calibre.service).

But how would all this work if I was using Dropbox? Dropbox shares files, it doesn't run services. So I'd expect the Calibre UI on my desktops and the Caliber Companions on my decides to access the files directly. So I'd not be running calibre.service at all.

But it's there a way to run calibredb to add books to a Calibre repository on Dropbox?

[kovidgoyal]

Use the calibre content server and you can add books, read them, edit their metadata etc directly in your browser on your mobile device. As for using calibredb with dropbox, calibredb does not care what your library is stored in as long as it is available as files on the disk, or via the content server (calibredb can connect directly to the content server)

[jdege]

Which is the Calibre content server?

Calibre-server?

[kovidgoyal]

https://manual.calibre-ebook.com/server.html

[jdege]

You mean the Content Server option within the Calibre UI?

That's of no use to me at all.

If I'm sitting in the lobby of a hotel, out of town, and decide I want to read a book that I've loaded into my Calibre repository, I'm not going to go back home, start the UI, and click the "Connect/share" button.

So I have calibre-server configured to run at start, as a background service, so it's up and running regardless of whether anyone is logged into the computer.

When I set this up, it was not possible to run both the Calibre UI and calibre-server at the same time, against the same repository, which is why I've been using a script to shutdown the server when I started the UI, and to start it back up on exit.

I'm doing the same with calibredb, because when I set it up I didn't see how to get it to talk to the server. Though from the documentation it looks like this is now possible.

But none of this is relevant to my question.

Calibre Companion has supported, for some time, a "Cloud" option, where you put your Calibre repository in a file sharing service like DropBox. I'm considering using this, instead of calibre-server.

But for this, I'd want to be able to run the Calibre UI against the file sharing service, and to be able to use calibredb to add books to the repository on DropBox.

DropBox mounts a directory in the local filesystem that mirrors the shared files. Changes to files within this directory are synced with the shared files on DropBox.

So it's clear to me that I could run both the Calibre UI and calibredb against this directory - but it's not clear that I'd not get consistency issues, as DropBox syncs the underlying files without knowing where the transaction boundaries are.

Is this something you've experimented with?

Any particular advice?

[DNSB]

Originally Posted by jdege

But for this, I'd want to be able to run the Calibre UI against the file sharing service, and to be able to use calibredb to add books to the repository on DropBox.

DropBox mounts a directory in the local filesystem that mirrors the shared files. Changes to files within this directory are synced with the shared files on DropBox.

So it's clear to me that I could run both the Calibre UI and calibredb against this directory - but it's not clear that I'd not get consistency issues, as DropBox syncs the underlying files without knowing where the transaction boundaries are.

Is this something you've experimented with?

Any particular advice?

Basically, the same advice most people would offer when it comes to running calibre with a network drive--don't! Most network drives do not offer the data locking support needed for concurrency & consistency.

A couple of people have reported success by stopping Dropbox sync, running calibre and restarting Dropbox sync after exiting calibre. However with calibre server being able to modify the database, you would need to stop Dropbox syncing before launching the server which defeats the point of the exercise. My solution was to run calibre server on my local machine/library using NATting on the router to allow access to calibre server from an external address.

You might want to check the calibre FAQs for I am getting errors with my calibre library on a networked drive/NAS? for more information.

[stumped]

You can forget about being able to do new stuff via calibre companion, because calibre companion access to Dropbox for new devices is broken and new developer has no intention of fixing it.

But have you considered setting up a folder in Dropbox which is configured as an add books folder for calibre, so anything you upload to that folder, from any device with Dropbox access, will be synced by Dropbox to the local folders of the device where calibre is running, and thus auto added to calibre
It should be possible to tell calibre that C:\myuseraccount\dropbox\intake. Or whatever is an auto add folder, and any Dropbox app or logged in browser can write to it

[jdege]

Originally Posted by DNSB

Basically, the same advice most people would offer when it comes to running calibre with a network drive--don't! Most network drives do not offer the data locking support needed for concurrency & consistency.

I was wondering if I could eliminate the need to keep my Linux system running. It looks as if I cannot.

[jdege]

Originally Posted by stumped

You can forget about being able to do new stuff via calibre companion, because calibre companion access to Dropbox for new devices is broken and new developer has no intention of fixing it.

But have you considered setting up a folder in Dropbox which is configured as an add books folder for calibre, so anything you upload to that folder, from any device with Dropbox access, will be synced by Dropbox to the local folders of the device where calibre is running, and thus auto added to calibre
It should be possible to tell calibre that C:\myuseraccount\dropbox\intake. Or whatever is an auto add folder, and any Dropbox app or logged in browser can write to it

If I'm going to continue to run calibre-server on my Linux box, I could continue to upload files with sftp, and ssh in to add them to the database.

But using DropBox as an auto-add sounds like it'd be easier.

I'm in the middle of configuring a new Linux box, and moving services over from the old, which is why I'm revisiting it. It looks like I'll be keeping my same structured with respect to Calibre.

Thanks, everyone, for the ideas and help.

[stumped]

Originally Posted by jdege

I was wondering if I could eliminate the need to keep my Linux system running. It looks as if I cannot.

that's where having a a mirror of library in a cloud is useful, it means i can wirelessly get books when main PC is off

but does linux have a wake on LAN or similar option so that you could remotely start it over internet with correct credentials and knowlegde of network card MAC, log in, do stuff, close it?

if like me, you don't actually power OFF, just leave it sleeping, then remote wake may be possible

[jdege]

My Linux box in running around the clock, except that it's actually a virtual, running on a Linux host, and the combination doesn't reliably restart after a power failure.

[Tech Guy]

I keep a copy of my Calibre library on Dropbox. My real library is on my local system, and I have a simple rsync script to make the Dropbox library the same as my local library, so anyone can do anything to the Dropbox library, and it will get fixed next time I run the rsync.

I use Calibre Companion on my Android phone, and it works very well. There was a problem a while back with new installs of the app, which I ran into when I got a new phone, but there has been a workaround published way back that worked fine for me - unless there is a new problem that I don't know about.

If you share you library, you need to be slightly careful about writing to it from more than one user. Mine is not often written to by others, and I've never had a problem except when a friend who didn't know what he was doing somehow erased all the books. Good think I had my copy. That's why I do the rsync all the time to add books. Nobody else should be adding books without telling me, but my rsync command first does a dry run, so I can see what changes will be made.

[stumped]

The Dropbox workaround only works for Gmail logins. I can't use that as my Gmail is linked to a different much smaller Dropbox, too small for my calibre library.

Trouble is that cc does not work well with other cloud solutions. Amazon dropped support, I could not get Google drive to work... Pcloud is not supported. The whole cloud area needs an overhaul but I feel it will never happen

I agree that the android app is fine for connecting to a pc running calibre. But if the new developer tries to enforce a subsription model I will do without it