Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 01-14-2011, 09:54 AM   #1
nickdma
Enthusiast
nickdma began at the beginning.
 
Posts: 41
Karma: 12
Join Date: Oct 2010
Device: Tablet PC
Managing multiples users and devices, ideas?

I've currently got 4 users, 4 computers, 4+ ereaders, and I'm trying to maintain 1 library across them.

I'm the "librarian", the rest of the family just uses the library to find what they want and put it on their readers. So I'm in search of ways to manage this.

One of the 4 computers is my "server", which runs the content server. My original goal was to put the main library on that computer, and share it out via the content server and NAS.

With 4K+ books, using the content server makes it a little tedious to browse the library looking for something. So the full client has become the preferred front end, especially for managing titles on the readers.

Calibre won't run in a "read-only" mode, if it doesn't have write access to the db, it fails to start. I don't want other clients to have write access to the db, since there's no db sharing built into calibre. I also don't want others mucking with the metadata, or deleting books. So I can't just point the calibre clients to the NAS share.

My initial solution, was to put a library on each PC, make it available on the LAN, and use the "copy to library" functionality. That worked, when it was just 2 libraries. Now the whole family is hooked on ereaders, and it's become a hassle. It's even worse if I change a bunch of metadata, there's no way to get that back to the other libraries.

I've thought about Dropbox, but it's two-way sync means at some point, two people will run calibre at the same time and trash the db. I'm looking at LiveMesh, which seems to have controls for who can make changes, but I suspect if I say it's read-only to members of the family, I'll be back to calibre not starting again.

Ideally, I think I'd like something like Dropbox, that syncs from the server to the clients, but not back. That puts all the data locally, so it performs better than the NAS. I don't need a "cloud" solution, but it has to be "drop box" simple. I'm still looking for solutions, some sort of rsync tool may suffice, if the simplicity is there.

If calibre supported a read-only mode (wink wink), that would help. NAS performance is getting better, but I had horrible issues with "folder device" performance. I'm not sure if we'd see the same with Nooks, I'd have to test that. Even if the NAS is too slow, a read-only mode may allow something like a read-only Live Mesh to work.

I know it would be a fair bit of work, but it would be nice if someday Calibre's content server could expand to act as a full calibre server, turning calibre into a client/server capable system. I'm not sure there's enough demand for such capability yet, but based on searches, I don't seem to be the only one trying to manage this.

Anyone have any creative solutions to sharing 1 master library to multiple PC's?
nickdma is offline   Reply With Quote
Old 01-14-2011, 10:04 AM   #2
chaley
"chaley", not "charley"
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 7,659
Karma: 1971516
Join Date: Jan 2010
Location: France
Device: Many android devices
If you are the only one making metadata changes, what is wrong with duplicating the library (rsync)? If your family makes changes (they aren't supposed to, apparently), they will be tossed next time you do the copy. It won't take long for them to understand that changing the library isn't productive.

Calibre cannot support a read-only mode. Python does not offer read-only access to an SQLite database.
chaley is offline   Reply With Quote
Old 01-14-2011, 11:35 AM   #3
nickdma
Enthusiast
nickdma began at the beginning.
 
Posts: 41
Karma: 12
Join Date: Oct 2010
Device: Tablet PC
The only problem with rsync (and most sync solutions I've found so far), is that it's not automatic and transparent.

I'm trying to find something that works like DropBox. Sync's just happen.

It's been my experience that schedule based tasks (AV updates, scans, etc) aren't that reliable when you have no idea when the clients will be online. Schedules always seem to get missed.

Apps that use more dynamic scheduling seem to work much more reliably, and transparent to the users. DropBox, CrashPlan, MS Security Essentials (although it often goes orange on the clients because scans are scheduled).

LiveMesh is out, because as far as I can tell, it doesn't support XP, which 2 of the compters use still.

I've found lots of nice tools, but they all require scheduling, or they're interactive. Technically, they'll work. I'm just hoping to find a more elegant solution.

I think I have the workflow sorted out... the library on my laptop becomes the master copy. I'll sync from my computer to the server, which will keep it, and the content server up to date. I'll configure the other computers to pull changes from the server, so they're not dependent on my laptop being online.

This also allows me to control when changes get "pushed" to everyone else.

I think it's just a matter of finding a good sync solution now. The search continues....
nickdma is offline   Reply With Quote
Old 01-14-2011, 01:42 PM   #4
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,244
Karma: 901265
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
I know that SugarSync (a Dropbox competitor) has the ability to apply permissions when sharing folders between different users. I have seen mention in the User Guide about only allowing some users read access to the folders/files - but whether this would stop them working with Calibre I am not sure. Therefore, whether this would solve your problem I do not know, but it might be worth checking out. If it all works with Calibre it should give you the sort of transparent syncing you are looking for.
itimpi is offline   Reply With Quote
Old 01-14-2011, 01:52 PM   #5
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 29,065
Karma: 7373157
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
you need to use rsync in pull rather than push mode. i.e. have it run from the client computers on every calibre startup. Simple to do by changing the calibre icon to run a bat file
kovidgoyal is online now   Reply With Quote
Old 01-14-2011, 04:03 PM   #6
nickdma
Enthusiast
nickdma began at the beginning.
 
Posts: 41
Karma: 12
Join Date: Oct 2010
Device: Tablet PC
That's the plan, to have the clients run in pull mode, with the exception of mine, which I will manually push to the server after I've made changes. I have to shut the content server down before doing the sync so the metadata.db can be overwritten.

So far, I've been using FreeFileSync (http://freefilesync.sourceforge.net/), which isn't quite what I was looking for, but it's getting the job done, and has a helper tool which will watch a folder for changes and perform a sync if I opt to automate it further.

I like the idea of hooking into the calibre startup. I have to see how often the library will change. Right now, it's changing alot, since I'm still cleaning up and adding/removing things. I suspect once it settles down, it might not be worth doing a scan at each startup on the clients.
nickdma is offline   Reply With Quote
Old 01-19-2011, 12:44 PM   #7
meme
Sigil developer
meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.
 
Posts: 1,275
Karma: 1101600
Join Date: Jan 2011
Location: UK
Device: Kindle PW, K4 NT, K3, Kobo Touch
I just got hit by this. Using Calibre on 2 Ubuntu systems with Dropbox. One is my main system, the other one is my wife's which I occasionally borrow. I did a number of edits on the main system, then went and started up Calibre on the second system - I don't really recall if I saved anything or not, or if I shutdown Calibre on the main system (it tends to stay running for the content server and because there's no reason to stop it).

Anyway, when looking at Calibre on the main system I noticed it lost most of my changes from yesterday. At first I decided to just recreate the updates, but then remembered its Dropbox - and it had a copy of the main systems' metadata.db file. I put that back in place and got all my updates back. My fault for using it this way as I know Calibre isn't meant to be used like this, but good to have the backups

So it might be good to have a 'start Calibre in read-only mode' option. I could setup an rsync pull, but as was said above, Dropbox just works (though a cron job is pretty trivial). With the option, I could even start it up with write enabled to do some edits on the second system if needed, but that's probably going to lead to problems again Maybe I'll look at VNC and rsync again...
meme is offline   Reply With Quote
Old 01-21-2011, 11:48 AM   #8
nickdma
Enthusiast
nickdma began at the beginning.
 
Posts: 41
Karma: 12
Join Date: Oct 2010
Device: Tablet PC
Quote:
Originally Posted by itimpi View Post
I know that SugarSync (a Dropbox competitor) has the ability to apply permissions when sharing folders between different users. I have seen mention in the User Guide about only allowing some users read access to the folders/files
I had initially passed on SugarSync because I thought it was limited to 2g for the free version, like DropBox. I just found out they give you a generous 5g, +500M referral bonuses, so I figured I'd try it out.

Sadly, while they offer a "read only" share, they don't offer a one-way sync. The read only share also doesn't actually sync any files to the other PC's. It simply makes them available to be downloaded through the SugarSync's app or web.
nickdma is offline   Reply With Quote
Old 01-26-2011, 12:21 PM   #9
loyukfai
Zealot
loyukfai has a spectacular aura aboutloyukfai has a spectacular aura aboutloyukfai has a spectacular aura aboutloyukfai has a spectacular aura aboutloyukfai has a spectacular aura aboutloyukfai has a spectacular aura aboutloyukfai has a spectacular aura aboutloyukfai has a spectacular aura aboutloyukfai has a spectacular aura aboutloyukfai has a spectacular aura aboutloyukfai has a spectacular aura about
 
Posts: 106
Karma: 4260
Join Date: Jan 2011
Device: none
I've been contemplating on switching to ebooks, and may setup a central Calibre server in future. One issue I still haven't figured out is that how can each person's notes and annotations should be handled.

As much as the convenience of using a single service like Kindle and Nook provides, I don't like the aspect that books may not be passable from one generation to the next, and sharing between family members is still something to be developed. Moreover, the thought of having your book collection controlled by a corporation which can delete your books anytime it likes doesn't sit well with me.

We need a better way to standardize, store and manage the notes and annotations, so it seems.

Cheers.

Edit: Seems the EPUB working group is working on annotation-related stuffs for the next revision.

http://code.google.com/p/epub-revision/wiki/Annotations

Last edited by loyukfai; 01-30-2011 at 07:39 AM.
loyukfai is offline   Reply With Quote
Old 02-12-2011, 02:25 PM   #10
cpenner
Junior Member
cpenner began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Feb 2011
Device: sony prs300/350
I just stumbled across this thread. I'm just starting down this road myself. I already manage my media library with boxee and a WD NAS device. I'm looking for a similar solution for my growing ebook library.

My current intention is to just put the Calibre library up on a NAS share, and have people only log into one client at a time. I'll keep watching this thread for people with better ideas...
cpenner is offline   Reply With Quote
Old 02-12-2011, 07:14 PM   #11
Gary_M_Mugford
Zealot
Gary_M_Mugford has a complete set of Star Wars action figures.Gary_M_Mugford has a complete set of Star Wars action figures.Gary_M_Mugford has a complete set of Star Wars action figures.
 
Gary_M_Mugford's Avatar
 
Posts: 136
Karma: 297
Join Date: Jul 2010
Location: Brampton ON
Device: Kobo, Kindle3
Might be speaking out of turn here and possibly out of my butt, but maybe what I do can help ...

My library sits on drive X: and is backed up nightly using Toucan to drive O: on one of the NAS's, where it is available for use (and misuse) by the rest of the readers here at the Castle of Confusion. THEY use a Toucan script that runs about 45 minutes after mine to backup the O: copy to their local data drives (it varies). All of this occurs at those odd hours of the morning when I'm sleeping. The result is a mirror copy, replete with my updates on their machines each morning and I have backups upon backups upon backups.

Although this does enforce a wait of up to 23 hours for the library to be updated, doesn't it do what you need? Toucan can be set to Mirror mode so that each destination folder becomes a mirror image of the source folder. The initial setup was a bit of a time-taker. But the changes are rare enough (I add books on Saturdays and make edits here and there through the week), that the Toucan script runs fairly quickly. Toucan's a Windows program and is available at http://portableapps.com/apps/utilities/toucan

One other thing I do in the batch file that Scheduler calls is backup the actual database file. This obviously requires calibre NOT to be running at the time, so adjust the batchfile as necessary in those circumstances.
Code:
@echo off
REM cycle backup of metadata
X:
cd \GaryELib
if exist metadata.db.7 del metadata.db.7
if exist metadata.db.6 ren metadata.db.6 metadata.db.7
if exist metadata.db.5 ren metadata.db.5 metadata.db.6
if exist metadata.db.4 ren metadata.db.4 metadata.db.5
if exist metadata.db.3 ren metadata.db.3 metadata.db.4
if exist metadata.db.2 ren metadata.db.2 metadata.db.3
if exist metadata.db.1 ren metadata.db.1 metadata.db.2
if exist metadata.db copy metadata.db metadata.db.1
E:
cd \apps\backups\ToucanPAF\Toucan.exe --job="xELib2oELib"
In effect, the original data is read-only to the rest of the family because they don't have access to it. They can experiment all they want (and report results) on their private copy of the data, safe in the knowledge nothing they are going to do is going to last past the next morning. You get backups aplenty (I have to admit, Carbonite Pro for 55 bucks a year makes for a secondary level of peaceful sleep) and peace of mind even that the odd accidental delete isn't 'permanent.'

Just a thought run amok, GM
Gary_M_Mugford is offline   Reply With Quote
Old 02-12-2011, 07:50 PM   #12
Stumbl
Thread Killer
Stumbl began at the beginning.
 
Stumbl's Avatar
 
Posts: 20
Karma: 10
Join Date: Oct 2010
Location: Myrtle Beach, SC. USA
Device: iPad
So that I can use calibre on my laptop OR on the desktop , I simply mapped a network drive pointing to the desktop from the laptop.
then I pointed calibre to the same library folder
your mileage may vary
Stumbl is offline   Reply With Quote
Old 02-18-2011, 12:47 AM   #13
cpenner
Junior Member
cpenner began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Feb 2011
Device: sony prs300/350
Quote:
Originally Posted by Gary_M_Mugford View Post
In effect, the original data is read-only to the rest of the family because they don't have access to it. They can experiment all they want (and report results) on their private copy of the data, safe in the knowledge nothing they are going to do is going to last past the next morning. You get backups aplenty (I have to admit, Carbonite Pro for 55 bucks a year makes for a secondary level of peaceful sleep) and peace of mind even that the odd accidental delete isn't 'permanent.'

Just a thought run amok, GM
That's one of the ideas I thought about. Essentially keeping one Master library and rsyncing it around the network. The big downside to this is that most of the client machines are laptops and are not usually "on" at scheduled times. I could have them sync just before launching Calibre, but that is a "hassle" step that will get the technology looked down on as "cludgy".

But most importantly - where did you get Carbonite Pro for $55 a year!
cpenner is offline   Reply With Quote
Old 02-18-2011, 01:57 AM   #14
Lady Fitzgerald
Wizard
Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.
 
Lady Fitzgerald's Avatar
 
Posts: 2,013
Karma: 251649
Join Date: Apr 2010
Location: Tempe, AZ, USA, Earth
Device: JetBook Lite (away from home) + 1 spare, 32" TV (at home)
Quote:
Originally Posted by cpenner View Post
That's one of the ideas I thought about. Essentially keeping one Master library and rsyncing it around the network. The big downside to this is that most of the client machines are laptops and are not usually "on" at scheduled times. I could have them sync just before launching Calibre, but that is a "hassle" step that will get the technology looked down on as "cludgy".

But most importantly - where did you get Carbonite Pro for $55 a year!
Methinks Mr. Mugford is talking about regular Carbonite. Pro is a business application that use tier pricing based on the amount of data stored. It starts at $10/mo/10GB and skyrockets from there. The regular Carbonite is $55/year for unlimited storage. Unlike the Pro version, it will backup only one computer and only drives within that computer. It will not backup servers nor external drives. I'm guessing it is also slower than the Pro version.

I also use Carbonite. It only backs up data files so it cannot be used to restore a computer like an image can but one can keep program installation files in a folder (you have to manually instruct Carbonite to back up program installation files) and use those to restore progams if necessary to restore your computer because your local backup went south. Despite these restrictions, Carbonite (Mozy is similar at $60/year) is still the best thing I've found for a reasonable price that will make near instantaneous, off site back ups of your data with limited versioning.
Lady Fitzgerald is offline   Reply With Quote
Old 02-19-2011, 08:04 PM   #15
Gary_M_Mugford
Zealot
Gary_M_Mugford has a complete set of Star Wars action figures.Gary_M_Mugford has a complete set of Star Wars action figures.Gary_M_Mugford has a complete set of Star Wars action figures.
 
Gary_M_Mugford's Avatar
 
Posts: 136
Karma: 297
Join Date: Jul 2010
Location: Brampton ON
Device: Kobo, Kindle3
Indeed, I misspoke Pro. Although we do use Pro at one of my clients.

My backup regiment is fairly anal, involving 30 days of backup for the C: drive which is strictly Windows, seven days of backup for the data on D: and Monthly backups of the apps folders on E:, all being stored on NAS's. There are internal backups that give me nightly backups of the data on local drives in the same computer and on other computers in the house (which are doing the same routines). And I have differential software running to record changes that can be rolled back on the data drive. And Carbonite gives me peace of mind.

AND I'VE HAD TO USE THEM ALL. A virus once got through that required going back 23 days to find a nice clean unadulturated C:\ image.

Way back in the dawn of the computer age, I probably wrote the single most brilliant short story ever. Pulitzer Prize winning material without a doubt. And I hit a button that was supposed to save everything. It was right beside the little one that meant delete on my Apple ][. I was never ever able to recover that single copy of the story that I deleted. And I was never able to recreate it. (So you can't prove it WASN'T the most brilliant short story ever [G]). But I did learn data is a very precious thing that needs protecting ... and backing up to the limits of your ability.

GM
Gary_M_Mugford is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Managing apps on multiple devices HarryT Apple Devices 3 07-15-2010 06:04 AM


All times are GMT -4. The time now is 10:51 PM.


MobileRead.com is a privately owned, operated and funded community.