Content Server Library Selection (also vhosts/url error questions)

[mmseng]

Hi there. After recently uninstalling/reinstalling Calibre and completely overhauling my library I've been trying to get it to work correctly with apache vhosts again (e.g. calibre.mydomain.com). Just as I was about to give up banging my head over it and go back to using www.mydomain.com/calibre I restarted Calibre and huzzah! out comes 0.7.25 with better support for this. Lo and behold! My vhosts config works without problems now! (mostly) Thanks devs.

Anyway, since I rebooted the whole setup I've been trying to be careful to keep only non-DRM'd books with accurate metadata in my library. I've not been so successful with the DRM restriction (as I've exhausted myself trying to crack the restrictions on my OWN damn Amazon/Kindle store-bought books), but I decided to use a second library (a feature I wasn't even aware Calibre had) as a kind of holding pen for books I've acquired but have not yet DRM-stripped or updated metadata for.

After I finally got the content server working the way I want I realized that it was showing me books from my holding-pen library and not my actual library. Apparently the content server only shows you books from the currently-active library. Is this correct?

If so, is there a way to make the content server only show a certain library, regardless of which library you're looking at in Calibre? Alternatively, would there be a way to specify which library you want to view when accessing the content server? If not I propose these features as suggestions.

----------------------------------

Secondly, now that I've got Calibre playing nice with vhosts using the ProxyPass directive, I'm wondering if I actually have a correct vhosts configuration. I've tried so many different configurations of Rewrite and ProxyPass that I've mixed myself all up. Here is what I have configured in Apache:

<VirtualHost *:80>
    ServerName calibre.mydomain.com
	ServerAlias cal.mydomain.com
	
	RewriteEngine On
	
	RewriteCond %{HTTP_HOST} ^cal\.mydomain\.com [NC]
	RewriteCond %{SERVER_PORT} ^80$
	RewriteRule ^(.*)$ http://calibre.mydomain.com$1 [L,R=301]
	
	<Location />
		ProxyPass http://calibre.mydomain.local:8085/
		ProxyPassReverse http://calibre.mydomain.local:8085/
	</Location>
</VirtualHost>

where, obviously, calibre.mydomain.com is my webservers external IP, and calibre.mydomain.local:8085 is the local address of Calibre's content server. The rewrite in use is purely to redirect cal.mydomain.com to calibre.mydomain.com (simply implementing best practices regarding domain canonicalization). Does this look right? The main reason I ask, is because I'm still getting a 502 (bad gateway) errors occasionally, seemingly randomly, when accessing different content server pages, and I don't know why.

----------------------------------

Thirdly, I used to have the following Rewrite directives (just above the ProxyPass chunk):

	RewriteCond %{REQUEST_FILENAME} !-f
	RewriteCond %{REQUEST_URI} !(.*)/$
	RewriteCond %{REQUEST_URI} !(.*)\.(.*)$
	RewriteRule ^(.*)$ http://calibre.mattseng.com$1/ [L,R=301]

which I use is several other vhosts for URL canonicalization. It takes URLs like www.mydomain.com/whatever and rewrites them as www.mydomain.com/whatever/, assuming the requested file does not include a period (which would indicate the request is for a file and not a directory).

Now, I used to use this with the old content server web interface, just because I used it in all my vhosts, even though you should really never be requesting a specific file or directory. However, now that the new web interface uses things like "/mobile", and "/browse/search?query=xxx", these directives actually break the content server. Why? Because it can handle requests for "/browse" and "/mobile", but apparently not "/browse/" or "/mobile/", which is what these directive would return when requesting without the trailing slash.

When you request a page with a trailing slash, or any non-existent page for that matter, the content server throws a (very ugly) 404:

404 Not Found

The path '/ajwnd' was not found.

Traceback (most recent call last):
File "site-packages\cherrypy\_cprequest.py", line 606, in respond
File "site-packages\cherrypy\_cperror.py", line 227, in __call__
NotFound: (404, "The path '/browse/' was not found.")

Powered by CherryPy 3.1.2

Obviously a 404 is normal for missing resources, and it only makes sense if /browse is an actual file and not a directory (I don't actually know the internal structure of the content server files). But it seems to me that, ideally, a request for "/browse/" should be either exactly, or at least functionally equivalent to a request for "/browse". A penny for your thoughts on this.

P.S. Oh, and both Apache and Calibre are running on the same Win2008 64-bit system.

Thanks for reading what turned out to be a ridiculously long post, and more thanks in advance for any help.
== Matt

[kovidgoyal]

Run the command server as a separate process from the command line as

calibre-server --with-library /path/to/library

You dont need both proxypass and rewriterule, see

http://calibre-ebook.com/user_manual/server.html

[mmseng]

Funny, I read that but somehow totally forgot about using the commandline server. Am I to understand that the commandline server functions differently from the built-in content server in regards to how it might communicate with my Apache webserver?

Also, when using the commandline server I assume you run Calibre without it's built-in server, and that they both simply access the same book database. Is that right? I'll try it out right now.

I don't really understand your comment about not using both rewrite and proxypass. My rewrite rules do not serve the purpose of changing the access url.

[mmseng]

Thanks for the reminder. The content server seems to be doing much better using the commandline version, in terms of misfires. I did not come across the calibre-server manual page (with usage) until just now. Previously I had even searched the word "calibre-server" in the online manual and it found nothing, weird. On top of allowing only a specified library I can even use it with Firedaemon this way! Sweet.

[kovidgoyal]

It's the exact same server code in both and yes you should turn off the GUI server if you are running the command line server. And I mean that you dont need to use proxypass you can just use a rewrite rule (well two actually to integrate with the calibre server)

[kovidgoyal]

The traceback is to help me debug problems. I know the conventional wisdom is to never show a user a traceback, but IMO that conventional wisdom makes about as much sense as the conventional wisdom about the earth being flat.

[kovidgoyal]

Incidentally, why do you have URL normalization that adds a trailing slash? I can understand normalization that removes trailing slashes, but why add them?

[mmseng]

Originally Posted by kovidgoyal

It's the exact same server code in both and yes you should turn off the GUI server if you are running the command line server. And I mean that you dont need to use proxypass you can just use a rewrite rule (well two actually to integrate with the calibre server)

There's no problem running the Calibre GUI and editing the database while the commandline server is running right? Just double checking...

Originally Posted by kovidgoyal

The traceback is to help me debug problems. I know the conventional wisdom is to never show a user a traceback, but IMO that conventional wisdom makes about as much sense as the conventional wisdom about the earth being flat.

Being a lower-level programmer myself I understand. It's just unexpected because of the "conventional wisdom", and to the untrained eye it looks more like a server side error than a standard 404. I guess I'm just a fan of making things pretty and the least confusing as possible to visitors.

Originally Posted by kovidgoyal

Incidentally, why do you have URL normalization that adds a trailing slash? I can understand normalization that removes trailing slashes, but why add them?

When I make websites I almost always use pages such as index.html or index.php which, obviously, most webservers autoload when a directory is requested that contains one of those files. I find the look of www.mydomain.com/store/, for example, to be much cleaner than www.mydomain.com/store.php or www.mydomain.com/store/index.php. And www.mydomain.com/store really isn't a valid resource (there's no non-extension'd file called "store" in the root directory). It's also a much more ergonomic solution for end users. All they have to do is go to mydomain.com/somepage and the server will automatically canonicalize it to www.mydomain.com/somepage/, obviously with SEO in mind.

Could I use the full file URL (www.mydomain.com/store/index.php) as the SEO-canon URL instead? Yes, but you can only have it one way (assuming you want canonical URLs), and that's the way I chose. Seems like it tends to be the more popular option as well.

To be perfectly honest I don't understand exactly what is being requested when I go to calibre.mydomain.com/browse. Is it a directory with an autoloaded index page? I figure it must be because if it were an actual script file I don't know how the webserver would know how to parse it. I'm mostly self-taught in all of this, so I'm sure some of my knowledge could stand to be corrected.

[kovidgoyal]

yes, you can, the command line server automatically detects changes to the database.

The calibre server has no concept of files or directories. It is fully virtual. It just uses names, so

http://localhost:8080/browse

in which case trailing slashes are pointless, they just mean more to type. In some cases it will simulate filenames so that the browser can download a file, but that's about it. I suggest turning off your canonicalization for the content server

[mmseng]

Originally Posted by kovidgoyal

The calibre server has no concept of files or directories. It is fully virtual.

Interesting. I've not come across something like that before. I'm just used to dealing with your standard hosted files.

So yeah I can see why slash rewrites wouldn't work at all for that.

Thanks for your help and for a great product.