Calibre Client Server Architecture

[creeva]

I'm trying to figure out if there is a method to use calibre as a client and server. Let me give you an example:

I have a large ebook library that I have installed with Calibre on a centralized server in my home. What I would like to do is open up Calibre on my laptop and browse the books on the centralized and import them into the library on my laptop.

I do know I can use the web page portion of the server to do this, but then I have to browse, download, import. I'm trying to get away from using the import process and using this method for all my computers in the house.

Ebooks seem to be one of the last steps to centralizing my media in this method. For audio I do this with itunes. For videos I use XBMC with a centralized SQL server. I'm really hoping there is something like this I can do for ebooks.

If this can't be done - what are some other ODS clients (other than stanza) that may work for me?

[mrmikel]

Someone will be along eventually who really knows what they are talking about, but here is what I understand. Calibre is not designed to be networked. Work is proceeding along those lines, but is not there yet. There are definite issues with accessing the database with different operating systems which do or do not consider capitals in file names significant. Fiddling around with the calibre database and its files outside of calibre is all but guaranteed to cause corruption because the program was not designed to create a standalone database. You might be able to organize files with calibre and export the results and use the results saved separately any way you wish.

Wish I could help you with the rest, but that is as far as my understanding runs.

[mwerle]

Originally Posted by creeva

I have a large ebook library that I have installed with Calibre on a centralized server in my home. What I would like to do is open up Calibre on my laptop and browse the books on the centralized and import them into the library on my laptop.

As long as you can mount the location of the centralised Calibre database on your laptop (Samba, NFS, Windows file sharing, whatever), this works fine.

Fire up Calibre on your laptop, create a new Library on the laptop if you haven't got one already, then open the existing Library on your server. You can now highlight one or more books, and send them to the Library on your laptop via a right-click.

There are two things to be careful about:

1. Ensure all versions of Calibre which access the Library on the server are the same version. While a newer version won't cause problems (the Library will be automatically upgraded), an older one might.
2. Only one device can access the central library on the server at a time, otherwise it can get corrupted.

I do this all the time While I do hope that Calibre will eventually have proper client-server support, this shared access to a central Library works ok.

Cheers,
- Micha.

[theducks]

Originally Posted by creeva

I'm trying to figure out if there is a method to use calibre as a client and server. Let me give you an example:

I have a large ebook library that I have installed with Calibre on a centralized server in my home. What I would like to do is open up Calibre on my laptop and browse the books on the centralized and import them into the library on my laptop.

I do know I can use the web page portion of the server to do this, but then I have to browse, download, import. I'm trying to get away from using the import process and using this method for all my computers in the house.

Ebooks seem to be one of the last steps to centralizing my media in this method. For audio I do this with itunes. For videos I use XBMC with a centralized SQL server. I'm really hoping there is something like this I can do for ebooks.

If this can't be done - what are some other ODS clients (other than stanza) that may work for me?

I am confused as to why you want to replicate your (central) library on each local machine? Sort of defeats the purpose of a central server .
Calibre is a hybrid of a DB and a file system. IIRC that was done that way for performance reasons.

The DB application places and keeps track of the file(s) associated with the metadata record. Currently, this is a User application and there is nothing to ensure that only one user affects the file system portion.

ex. User 2 changes the author name spelling just as User 1 is adding a new title file to that author

[creeva]

I don't really want to share the database. I guess to put it simply - I want to run one main calibre server - I would like to have other installations act as ODS clients. I don't want my laptop to have every book in the main server - but I would like to use an interface that can browse the server and import books automatically.

With the stanza reader client you can do this, I would just like similar functionality to be an ODS client in calibre. It would also give the flexibility to browse an import books from at least all the sources that Stanza can.

[creeva]

Originally Posted by theducks

I am confused as to why you want to replicate your (central) library on each local machine? Sort of defeats the purpose of a central server .
Calibre is a hybrid of a DB and a file system. IIRC that was done that way for performance reasons.

The DB application places and keeps track of the file(s) associated with the metadata record. Currently, this is a User application and there is nothing to ensure that only one user affects the file system portion.

ex. User 2 changes the author name spelling just as User 1 is adding a new title file to that author

I don't want to replicate the whole database (or I would not have a central server) - I just want to be able to import books in directly by highlighting them and click import. The itunes home sharing model - or like i said in my other reply the way Stanza does it. This is the whole reason Calibre has the server function built in. I would just like it to finish the other half and act as a client also.

The MySQL suggestions through the threads that people are wanting isn't a completely bad idea - but it would take much work to port it properly to MySQL than it would be program an ODS client (ok to be fair that's a guess).

[ilovejedd]

First, I believe what you're wanting is an OPDS client and not ODS. Second, the EPUBReader add-on for Firefox does support OPDS catalog feeds. Useful if you just want to read books on your PC/laptop. If you actually need to copy the books to calibre for sending to device, etc, not so much.

I believe it's been mentioned before that there are plans to include server/client set-up for calibre but I wouldn't expect to see that anytime soon.

[creeva]

Originally Posted by ilovejedd

First, I believe what you're wanting is an OPDS client and not ODS. Second, the EPUBReader add-on for Firefox does support OPDS catalog feeds. Useful if you just want to read books on your PC/laptop. If you actually need to copy the books to calibre for sending to device, etc, not so much.

I believe it's been mentioned before that there are plans to include server/client set-up for calibre but I wouldn't expect to see that anytime soon.

Yes I think that's what I'm going over - and it seems that is the last step for Calibre to be feature complete. I don't use Firefox (over the last year I have become a Chrome person). I'm well aware there are readers that can act as a client - as I mentioned i do use Stanza. But from what I gather from the ethos of Calibre is that is about choice and freedom. Saying I need to use another software package to utilize a function within Calibre seems against that.

Calibre gives you the freedom to read books in whatever format you like. If you don't like a format you can change it to a different one. If you don't like the metadate from one source you can choose a different one. If you don't like standard RSS readers you can change them into a ebook and use them however you would like. If you want to transfer books to your device you can use a multiple of methods including email, mounted drive, or direct device interfacing. If you don't want to use a separate ereader you can use the built in ereader.

From my perspective having the software only act as a server and not as a client is just about the only function where you need a third party software either in the form of a web browser or reader that can attach to an ODS server. So for that function choice is taken away from me. It also makes taking what I want with me where a pain.

In our household there is a server, 2 windows desktops, 2 mac laptops, 6 iOS devices, and 2 kindles. Depending on which device I'm on is going to reflect which books I want with me at any given time.

For any media other than ebooks I already have good solutions for making things work between all devices. I am however glad it is on the roadmap, and I wish I was a programmer to help get it done.

It is just annoying to have to attach to a calibre server via web browser and download the book. Then turn around an import it into Calibre again.

[itimpi]

You do not have to load the book into Calibre if you simply want to read it. The Calibre viewer can be used as a free-standing reader. However there are lots of alternative iewers around.

[creeva]

Originally Posted by itimpi

You do not have to load the book into Calibre if you simply want to read it. The Calibre viewer can be used as a free-standing reader. However there are lots of alternative iewers around.

I understand that - my problem is not the viewer. If I wanted to remove calibre from the equation - after any conversion and metadata was done I would just write the books to a directory structure and then use them on a remote device. But once I've done that on a file server - then there isn't much use to calibre at all if I have find alternative ways for file storage on each and every device I own.

I look at calibre like itunes - a simple interface to manage a multitude of formats. I also have itunes installed on my server, my two desktops, and my two laptops. So I can share and copy music however I want - while maintaining a master library on my server.

I already understand the work arounds (and live with them) - since I have been using calibre for a couple years now. I wasn't aware it couldn't be used as a client and thought I was missing something. I still stand by the fact that an opds client is what stands in teh way of it being feature complete. The only other thing I could think of to add is a store interface.

[ilovejedd]

Originally Posted by creeva

I already understand the work arounds (and live with them) - since I have been using calibre for a couple years now. I wasn't aware it couldn't be used as a client and thought I was missing something. I still stand by the fact that an opds client is what stands in teh way of it being feature complete. The only other thing I could think of to add is a store interface.

I do believe there's already a storefront for various ebook sources.

I don't think calibre needs an OPDS client for what you want to accomplish. There are other methods for implementing a server/client set-up. Best is probably to take a similar route to XBMC (admittedly, I'm not sure how XBMC is set-up exactly as I use WMC+Media Browser). For ebooks, though, I'd probably prefer something similar to iTunes Home Sharing.

[creeva]

The XBMC method I can explain away. There are a few ways you can chain XBMC setups. The first is you have a server and from there other XBMC instances can access the libraries via UPNP. I was using this method for a while. I moved over to using a MySQL database where all the XBMC instances share the same library.

The only real advantage I have with using MySQL (in what actually works for me) is that I only have to scrape the media once and then it is available to all my machines.

For Calibre something like itunes home sharing would be perfect.

[atamido]

Originally Posted by creeva

I moved over to using a MySQL database where all the XBMC instances share the same library.

The only real advantage I have with using MySQL (in what actually works for me) is that I only have to scrape the media once and then it is available to all my machines.

To expand on using a MySQL database, the covers and such used in XBMC are placed on a share (SMB/CIFS/NFS/etc) so that if one system updates them, the change is available immediately to all systems.

Beyond being able to share your same database information and cover/content files, MySQL offers much greater speed when working with a large number of entries. We're talking an order of magnitude faster. There is just no practical way to design an embedded relational database like SQLite to have the performance of a dedicated relational database when you're dealing with a lot of data.

My ideal would be to have a central file store with a MySQL database that all of my families computers could access. And when they want to be on the go with a particular book, copy the book to their local library. That way their local libraries are small (and so fast), while the main library is large but still fast because of the full database system.

[creeva]

I'm just curious if anyone is even looking at coding this. I think the focus is on the client side.

[MikeTampa]

I know where you're coming from, feel much the same way. Except I don't think XBMC is the exactly paradigm we want this to head towards...we want it to be more like Plex.

I'm a former XBMC user, now converted. Plex is very similar, I think both share same origins. But Plex is true client server. It has small server component, web-based library management, and full-featured native clients for your myriad playback devices. Brand new clients can instantly access/use the entire library--the shared metadata is already there.

Ideally calibre could work in similar fashion to Plex. Instead of offering a web-based client (which is what we have now with the calibre server), better would be full-featured native clients, with web-based library management. I think the ratio of time one spends reading vs. managing the library supports such a split?

One requirement that's different for ebooks than video (Plex) is, I think, ready ability to make and manage local content copies on the client...for use when disconnected. I am not familiar with iTunes, don't know if it has this ability?