Calibre Server -- Multi Tenant

[dstrohl]

Note: I am pretty new to contributing to open source projects (though I have done development work here and there for a lone time), so if I have made any mistakes in approaches or social gaffs, I apologize in advance.

I have wanted to have the server support offering different sets of books to different users. In looking through this forum, I saw that this has come up at least a couple of time in the past, but it looks like it is a bit further down Kovid's to-do list, so I figured I would take a whack at it from the server side...

My thought is to approach this using the username password mechanism that is already in place. so, depending on what username was used, a different virtual library would be selected. I know this limits this feature to devices or browsers that allow username password for opds (I don't know how limiting this is though) but it seems like it is a fair first step.

I considered trying for a url pattern for this (if security was not an issue), but in poking in the code, that seemed like that would be harder to implement (thoughts?)

I also plan on implementing it in an open enough way that hopefully it could integrate with future multi-user plans.

My main question is one of approach, I am sure I can back myself into this, but it seems easier to simply ask here: Based on some of the ways I have seen different web applications work, there are probably a couple of approaches to try here:

1. if the server class is instantiated for each connection, then I can check the user and set the virtual library restrictions at that point (I think this looks easier to do).
-or-
2. if the server class is instantiated once, and connections are passed to it for content, I may have to check each request or re-filter for each request. (probably not hard, but I haven't played enough with the db api to really get a good feeling for how hard... or not... this would be)

Thoughts? (overall thoughts? is this a dumb project, or just the wrong way of going about it? or something that is already solved I just wasn't smart enough to find the answer?)

Dan

[eschwartz]

Well, currently you *could* start the server via the command-line, run multiple servers on multiple ports, and initialize each one with a different password and restriction.

This limits the usability especially for people not comfortable with scripting (like, well, the majority), so I say go for it. It sounds like a good thing to have.

[kovidgoyal]

There is only a single instance of the Server class. You would have to restrict the data you send per request. You would use the same mechanism as the current

--virtual-library

option to the server, except that the virtual library to use would now need to be gotten based on the authenticated user per request. There are various caching mechanisms in the server to speed up --virtual-library, which might have to be generalized as well. The database API is perfectly capable of returning data for different restrictions in the same isntance.

On a general note, calibre's content server is really designed to be a personal server not one for distribution to multiple parties. So while I wont refuse this functionality, it has to be implemented in a way that does not compromise the core use case.

[dstrohl]

Originally Posted by kovidgoyal

There is only a single instance of the Server class. You would have to restrict the data you send per request. You would use the same mechanism as the current

--virtual-library

option to the server, except that the virtual library to use would now need to be gotten based on the authenticated user per request. There are various caching mechanisms in the server to speed up --virtual-library, which might have to be generalized as well. The database API is perfectly capable of returning data for different restrictions in the same isntance.

Thanks, that is what it looked like, I was hoping for the other approach, but the more I poked through the code, the more it looked like that would be the case.

Originally Posted by kovidgoyal

On a general note, calibre's content server is really designed to be a personal server not one for distribution to multiple parties. So while I wont refuse this functionality, it has to be implemented in a way that does not compromise the core use case.

I agree there, there are many reasons that this would not be a good implementation for a larger scale server, starting (and and probably ending) with security. I am NOT trying to implement a real security system here, simply a way of "hiding" content from certain groups, or offering more focused content based on the login.

My thought is to use the following assumptions:

- If there are no tenants configured, everything operates just like normal:
- There would be a checkbox and command line switch to turn off the functionality even if they were configured, for troubleshooting if nothing else.
- if a user connects with no user credentials passed, it would default to the virtual library (or lack of one) defined in the normal ui. (so, if you want to make sure a user name was needed for everything, they would also have to configure the user info in the main screen.

This should have a pretty low impact on any existing systems, and make it easier to pull or fix later...

The basic use case for this that I am shooting for is something like a family where all the books are in the library, but the kids can see any of the fiction ones that don't have a tag "adults-only", spouse 1 (with a home business) can see all of the family stuff, plus some reference books for their business, and spouse 2 can see all of the family stuff as but not the business books.

I can also see it as a first pass filter, this would allow me to define a "server" in marvin that would publish just reference docs, and another one that just had the current fiction that I was interested in, and a third server that had everything.

In any case, the goal is not to go away from the idea of a personal server, just to broaden it a little. (make sense?)

Dan

[dstrohl]

So... based on the dev docs, I forked calibre to my own git-hub account (https://github.com/dstrohl/calibre) and made a branch for the multi-tenancy stuff...

I assume this is the preferred approach?

[kovidgoyal]

Sounds reasonable, I might have a few more comments when I see actual code, but in principle it sounds fine.

In particular, I anticipate there may be performance issues. A virtual library is just a search, and while the db api and the server both have caching infrastructures, they may not work well with different VLs per request.

Also, there will need to be some kind of UI for configuring this in Preferences->Sharing over the net.

Using a branch is fine.

[dstrohl]

One more quick question, in the options/preferences , is there a good example of saving a dynamic list of a _group_ of settings? (somewhat like the custom columns section, but that saves the config in the db it looks like, or the ui for plugboards, but much less complicated)?

I am not worried about -how- to do this, I can think of several ways, I just want to try to use the same underlying approaches that are used elsewhere when possible, and to not reinvent the wheel.

Dan

[dstrohl]

Originally Posted by kovidgoyal

In particular, I anticipate there may be performance issues. A virtual library is just a search, and while the db api and the server both have caching infrastructures, they may not work well with different VLs per request.

Yup, that is my expectation as well (at least in terms of where I expect problems). I have some thoughts, but until I get into the code I am not sure what will work and what wont.

Originally Posted by kovidgoyal

Also, there will need to be some kind of UI for configuring this in Preferences->Sharing over the net.

Also "yup", (I just posted a quick question about the best way to store this into in the prefs, I am trying not to change the structure of the preference storage for this so there are no upgrade/downgrade issues. this should have zero impact (or close to it) if not used.

Originally Posted by kovidgoyal

Using a branch is fine.

I already forked the repo, but I haven't done much, would you prefer that I just branch the code in your repo and move my changes there? (sorry about that, I thought I read that you wanted it forked)

[kovidgoyal]

A fork is fine. The branch simply referred to creating a branch in your own fork, which is optional, you can use the master branch if you like or create a dedicated branch, it makes no difference to me.

As for storing preferences, I suggest getting the code working with hard coded values in LibraryServer.__init__ first, once we have a better idea of how exactly it works we can worry about config for it.


Currently the server is designed for convenient use both from the GUI and from the command line. To maintain that with this functionality, you would probably need to pass a dict of username->(password, vl) to LibraryServer. When running from the GUI it can be stored as a JSONConfig object in calibre's usual preference hierarchy. When running from the command line, probably an option to read it from the specified file.

[eschwartz]

http://manual.calibre-ebook.com/deve...to-be-included