Abandoned attempt to use calibre with SSH mounted drive

astrodad · 10-03-2008, 09:56 AM

Some of you may know about my quest to maintain my calibre database in the cloud. I've tried everything. I had been trying to use an SSH-mounted drive to place my entire calibre database remotely. This sort of worked, but was always very slow. On Windows, using Sftpdrive, it was sort of okay, on a Mac with Expandrive (using MacFuse) it was terrible.

With the introduction of the latest version of calibre, wherein Kovid has changed the application to use a file system for the ebooks and a metadata database, things improved on Windows, but not on the Mac. Plus I would continue to get I/O errors, dropouts, and general instability.

After much frustration, I've given up that effort. Now I just use DropBox. I put the library in a 'library' folder on my Dropbox share and let it do its thing as I add/delete/update books. It's much easier, the application is now as responsive as it's supposed to be, and I don't have to worry about maintaining a constant network connection.

I would still like to see a version of calibre that implemented a more network-capable database driver, but for now I am happy!

The only problem I have with Dropbox is the 2GB barrier. I already have my own Amazon S3 and could use that, but Dropbox doesn't support custom buckets. Does anyone know of a background synch program that works with your own Amazon S3 account? Or a background synch program that works with SFTP?

pilotbob · 10-03-2008, 10:51 AM

Quote:

Originally Posted by jplumey

The only problem I have with Dropbox is the 2GB barrier. I already have my own Amazon S3 and could use that, but Dropbox doesn't support custom buckets. Does anyone know of a background synch program that works with your own Amazon S3 account? Or a background synch program that works with SFTP?

Have you looked a JungleDisk? It allows for mounting an S3 account as a drive on Windows and Mac.

BOb

kovidgoyal · 10-03-2008, 11:13 AM

I use unisync over sftp to keep share my library across my two primary computers.

astrodad · 10-03-2008, 11:19 AM

Quote:

Originally Posted by pilotbob

Have you looked a JungleDisk? It allows for mounting an S3 account as a drive on Windows and Mac.

BOb

I use JungleDisk as my primay net backup but it doesn't have a real-time sync. And using JungleDisk, mapped as a drive or mount point, is bad for calibre because the changes don't occur in real-time. They are cached and then uploaded in the background.

I tried it earlier and it was a colossal failure. It's a great backup solution though.

pilotbob · 10-03-2008, 11:23 AM

Quote:

Originally Posted by jplumey

I tried it earlier and it was a colossal failure. It's a great backup solution though.

Right. I thought you had said you tried it, but couldn't remember.

Here is something that I was looking at before I decided to go with TimeMachine on OSx. I may trial it just to keep my documents in sync between my PC and Mac, $24 a year for 10gb isn't to bad. They have a free trial for 30 days.

http://www.sugarsync.com/

Maybe you might want to give it a wack.

BOb

astrodad · 10-03-2008, 11:43 AM

Thanks for the suggestion. That looks about the same as Dropbox. What I would like is a way to sync to my own Amazon S3 or SSH-mounted remote drive. SugarSync and Dropbox are nearly the same in that they go to their own data repositories.

JeffElkins · 10-03-2008, 11:52 AM

I've been using an NFS drive for my database. No problems thus far.

kovidgoyal · 10-03-2008, 05:00 PM

The simplest non-real time solution (provided you can be sure that only one person is going to be using the central database location is to create a simple wrapper script for launching calibre with rsync)

Code:

#!/bin/sh
rsync myserver.net:/path/lo/library/folder /path/to/local/copy
calibre
rsync /path/to/local/copy myserver.net:/path/lo/library/folder

rsync will only copy changed files and it can use ssh as well as various other transport protocols.

pilotbob · 10-03-2008, 08:19 PM

Quote:

Originally Posted by kovidgoyal

Code:

#!/bin/sh
rsync myserver.net:/path/lo/library/folder /path/to/local/copy
calibre
rsync /path/to/local/copy myserver.net:/path/lo/library/folder

rsync will only copy changed files and it can use ssh as well as various other transport protocols.

That's a darned good idea. I'm going to do that. I assume rsync is platform agnostic? Now, to figure out how to do that on a mac rather than dragging the application to my doc or running from spotlight... hmmm. (still learning mac OS innards. I could do this on Windows with my eyes closed.)

BOb

BOb

astrodad · 10-06-2008, 09:46 AM

So far DropBox is working fantastic for me. I've used rsync in the past and I've never had problems with it, so I may look at that again. The problem is that I would have to set up a different process on each OS that I work with (Two Mac boxes, Windows, and Linux). Dropbox just work right now without having to do a bunch of configuration. But thanks for the suggestion!

10-03-2008, 09:56 AM	#1
astrodad Guru Posts: 962 Karma: 568242 Join Date: Dec 2007 Device: Kindle Oasis 3, Kindle Paperwhite 1, iPad, iPhone	Abandoned attempt to use calibre with SSH mounted drive Some of you may know about my quest to maintain my calibre database in the cloud. I've tried everything. I had been trying to use an SSH-mounted drive to place my entire calibre database remotely. This sort of worked, but was always very slow. On Windows, using Sftpdrive, it was sort of okay, on a Mac with Expandrive (using MacFuse) it was terrible. With the introduction of the latest version of calibre, wherein Kovid has changed the application to use a file system for the ebooks and a metadata database, things improved on Windows, but not on the Mac. Plus I would continue to get I/O errors, dropouts, and general instability. After much frustration, I've given up that effort. Now I just use DropBox. I put the library in a 'library' folder on my Dropbox share and let it do its thing as I add/delete/update books. It's much easier, the application is now as responsive as it's supposed to be, and I don't have to worry about maintaining a constant network connection. I would still like to see a version of calibre that implemented a more network-capable database driver, but for now I am happy! The only problem I have with Dropbox is the 2GB barrier. I already have my own Amazon S3 and could use that, but Dropbox doesn't support custom buckets. Does anyone know of a background synch program that works with your own Amazon S3 account? Or a background synch program that works with SFTP?

10-03-2008, 05:00 PM	#8
kovidgoyal creator of calibre Posts: 45,240 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	The simplest non-real time solution (provided you can be sure that only one person is going to be using the central database location is to create a simple wrapper script for launching calibre with rsync) Code: #!/bin/sh rsync myserver.net:/path/lo/library/folder /path/to/local/copy calibre rsync /path/to/local/copy myserver.net:/path/lo/library/folder rsync will only copy changed files and it can use ssh as well as various other transport protocols.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
backing up Calibre folder on USB drive	dcarlozzi	Related Tools	4	06-01-2010 08:59 AM
Calibre unable to detect drive on Mac	patchymama	Calibre	5	11-29-2009 11:40 AM
Conflict between Calibre and mounted TrueCrypt device volume	acidzebra	Calibre	4	08-13-2009 03:11 PM
Calibre on a flash drive	rcuadro	Calibre	5	02-13-2009 09:44 AM
Can the EBW1150 be mounted as a usb drive?	nrapallo	Fictionwise eBookwise	7	12-12-2008 04:53 AM

10-03-2008, 11:13 AM	#3
kovidgoyal creator of calibre Posts: 45,240 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I use unisync over sftp to keep share my library across my two primary computers.

10-03-2008, 11:43 AM	#6
astrodad Guru Posts: 962 Karma: 568242 Join Date: Dec 2007 Device: Kindle Oasis 3, Kindle Paperwhite 1, iPad, iPhone	Thanks for the suggestion. That looks about the same as Dropbox. What I would like is a way to sync to my own Amazon S3 or SSH-mounted remote drive. SugarSync and Dropbox are nearly the same in that they go to their own data repositories.

10-03-2008, 11:52 AM	#7
JeffElkins Guru Posts: 780 Karma: 1416 Join Date: Jan 2008 Device: Kobo Clara 2E/HD, Kindle PW	I've been using an NFS drive for my database. No problems thus far.

10-06-2008, 09:46 AM	#10
astrodad Guru Posts: 962 Karma: 568242 Join Date: Dec 2007 Device: Kindle Oasis 3, Kindle Paperwhite 1, iPad, iPhone	So far DropBox is working fantastic for me. I've used rsync in the past and I've never had problems with it, so I may look at that again. The problem is that I would have to set up a different process on each OS that I work with (Two Mac boxes, Windows, and Linux). Dropbox just work right now without having to do a bunch of configuration. But thanks for the suggestion!

Advert

Advert