Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Related Tools

Notices

Reply
 
Thread Tools Search this Thread
Old 03-03-2014, 11:57 AM   #1
DouglasK
Member
DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.
 
DouglasK's Avatar
 
Posts: 16
Karma: 7838
Join Date: Oct 2010
Location: Ottawa, Ontario, Canada
Device: Kobo Aura HD, Nexus 7 (2013)
Question Calibre In Process Server help

I'm seeing a 500 error from Apache trying to load it. I've had the in process server running before without issue (on Ubuntu Linux), but have recently had to rebuild the server due to a h/w failure. Version of Ubuntu is now 13.10, This updated the Apache server from 2.2 to 2.4, not sure if that's relevant.

Any help is appreciated.

In short, Calibre is saying it can't find calibre-wsgi-adapter.py. Here's (what I hope are) the relevant logs and configs:

Here's the error from the Apache logs:
Code:
[Mon Mar 03 10:45:27.971954 2014] [:error] [pid 6767] [client xx.xx.204.206:52282] mod_wsgi (pid=6767): Target WSGI script '/var/www/cgi-bin/calibre-wsgi-adapter.py' cannot be loaded as Python module.
[Mon Mar 03 10:45:27.971987 2014] [:error] [pid 6767] [client xx.xx.204.206:52282] mod_wsgi (pid=6767): Exception occurred processing WSGI script '/var/www/cgi-bin/calibre-wsgi-adapter.py'.
[Mon Mar 03 10:45:27.972012 2014] [:error] [pid 6767] [client xx.xx.204.206:52282] Traceback (most recent call last):
[Mon Mar 03 10:45:27.972058 2014] [:error] [pid 6767] [client xx.xx.204.206:52282]   File "/var/www/cgi-bin/calibre-wsgi-adapter.py", line 31, in <module>
[Mon Mar 03 10:45:27.972119 2014] [:error] [pid 6767] [client xx.xx.204.206:52282]     from calibre.library.server.main import create_wsgi_app
[Mon Mar 03 10:45:27.972142 2014] [:error] [pid 6767] [client xx.xx.204.206:52282] ImportError: No module named calibre.library.server.main
Here's the output of calibre-debug --paths:
Code:
$ calibre-debug --paths
CALIBRE_RESOURCES_PATH=/opt/calibre/resources
CALIBRE_EXTENSIONS_PATH=/opt/calibre/lib/python2.7/site-packages/calibre/plugins
CALIBRE_PYTHON_PATH=/opt/calibre/lib/python2.7:/opt/calibre/lib/python2.7/plat-linux2:/opt/calibre/lib/python2.7/lib-dynload:/opt/calibre/lib/python2.7/site-packages
Here's the content of calibre-wsgi-adapter.py:
Code:
# WSGI script file to run calibre content server as a WSGI app

import sys, os


# You can get the paths referenced here by running
# calibre-debug --paths
# on your server

# The first entry from CALIBRE_PYTHON_PATH
sys.path.insert(0, '/opt/calibre/lib/python2.7')

# CALIBRE_RESOURCES_PATH
sys.resources_location = '/opt/calibre/resources'

# CALIBRE_EXTENSIONS_PATH
sys.extensions_location = '/opt/calibre/lib/python2.7/site-packages/calibre/plugins'

# Path to directory containing calibre executables
sys.executables_location = '/opt/calibre'

# Path to a directory for which the server has read/write permissions
# calibre config will be stored here
os.environ['CALIBRE_CONFIG_DIRECTORY'] = '/var/www/books/calibre-config'

del sys
del os

from calibre.library.server.main import create_wsgi_app
application = create_wsgi_app(
        # The mount point of this WSGI application (i.e. the first argument to
        # the WSGIScriptAlias directive). Set to empty string is mounted at /
        prefix='/calibre',

        # Path to the calibre library to be served
        # The server process must have write permission for all files/dirs
        # in this directory or BAD things will happen
        path_to_library='/home/douglask/CalibreWeb',

        # The virtual library (restriction) to be used when serving this
        # library.
        virtual_library=None
)

del create_wsgi_app
DouglasK is offline   Reply With Quote
Old 03-03-2014, 08:11 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,154
Karma: 5381911
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
You cannot use the in process server with a binary calibre install, as stated in the User Manual. Use the reverse proxy.
kovidgoyal is offline   Reply With Quote
 
Enthusiast
Old 03-05-2014, 12:21 PM   #3
DouglasK
Member
DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.
 
DouglasK's Avatar
 
Posts: 16
Karma: 7838
Join Date: Oct 2010
Location: Ottawa, Ontario, Canada
Device: Kobo Aura HD, Nexus 7 (2013)
Quote:
Originally Posted by kovidgoyal View Post
You cannot use the in process server with a binary calibre install, as stated in the User Manual. Use the reverse proxy.
Fair enough ... per the docs it should work with a current version installed by the package manager. I include the link to the docs page to ensure we're both referring to the same document.

I initially tried the version through the package manager, and have now reverted to that. The installed version is 1.22.0 .

Now what I'm seeing is the browser sitting on "Connecting". In the server error log (log level is set to debug) I'm seeing the following:
Code:
[Wed Mar 05 12:16:11.147060 2014] [authz_core:debug] [pid 13378] mod_authz_core.c(802): [client 66.46.204.206:44211] AH01626: authorization result of Require all granted: granted
[Wed Mar 05 12:16:11.147166 2014] [authz_core:debug] [pid 13378] mod_authz_core.c(802): [client 66.46.204.206:44211] AH01626: authorization result of <RequireAny>: granted
[Wed Mar 05 12:16:11.147535 2014] [authz_core:debug] [pid 13378] mod_authz_core.c(802): [client 66.46.204.206:44211] AH01626: authorization result of Require all granted: granted
[Wed Mar 05 12:16:11.147624 2014] [authz_core:debug] [pid 13378] mod_authz_core.c(802): [client 66.46.204.206:44211] AH01626: authorization result of <RequireAny>: granted
[Wed Mar 05 12:16:11.159324 2014] [:info] [pid 13378] mod_wsgi (pid=13378): Create interpreter 'fog.org|/calibre'.
[Wed Mar 05 12:16:11.160495 2014] [:info] [pid 13378] [client 66.46.204.206:44211] mod_wsgi (pid=13378, process='', application='fog.org|/calibre'): Loading WSGI script '/var/www/cgi-bin/calibre-wsgi-adapter.py'.
Any guidance for getting the in process server working is appreciated.
DouglasK is offline   Reply With Quote
Old 03-05-2014, 12:28 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,154
Karma: 5381911
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I'm afraid I dont have the time to help, just use the reverse proxy. There is absolutely no advantage to using an in process server.
kovidgoyal is offline   Reply With Quote
Old 03-06-2014, 07:20 AM   #5
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 8,863
Karma: 12755553
Join Date: Feb 2009
Location: North Carolina
Device: Nexus 7
Quote:
Originally Posted by DouglasK View Post
Fair enough ... per the docs it should work with a current version installed by the package manager. I include the link to the docs page to ensure we're both referring to the same document.
...~~~...
Any guidance for getting the in process server working is appreciated.
Since you linked to the docs you must have seen this note.

Quote:
Note: For this to work, all the dependencies needed by calibre must be installed on your system. Doing so is highly non-trivial and you are encouraged not to use in process servers. You will not get any assistance with debugging in process server problems.
DoctorOhh is offline   Reply With Quote
Old 03-06-2014, 03:37 PM   #6
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
Posts: 6,683
Karma: 11160789
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
Quote:
Originally Posted by DoctorOhh View Post
Since you linked to the docs you must have seen this note.
Kovid just changed that. I'm pretty sure it was in response to this thread.

See: https://github.com/kovidgoyal/calibr...dbd346ba192890
eschwartz is offline   Reply With Quote
Old 03-06-2014, 04:48 PM   #7
DouglasK
Member
DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.DouglasK knows the square root of minus one.
 
DouglasK's Avatar
 
Posts: 16
Karma: 7838
Join Date: Oct 2010
Location: Ottawa, Ontario, Canada
Device: Kobo Aura HD, Nexus 7 (2013)
Quote:
Originally Posted by eschwartz View Post
Kovid just changed that. I'm pretty sure it was in response to this thread.

See: https://github.com/kovidgoyal/calibr...dbd346ba192890
Yup, it's a new addition to the page.

I'm suspecting that it has to do with the move from Apache 2.2 to Apache 2.4. The In Process server used to work just fine for me using the same distro packages. I've seen other mentions of folk having issues after moving to Apache 2.4 as well.

Here's why I'm not big on running the Calibre server and doing a reverse proxy: It keeps several processes running, each gobbling 115Mb (approx) of resident memory, and 3.3GByte of virtual memory. In my opinion, that's a heavy load to continually run on a server even when it's not being accessed. iirc, the in process server was a tad slower, but would load only on an as needed basis. To me, this is far preferable in a scenario where I'll access it primarily when I'm done a book and looking for something to read.

As for Kovaid, what, if any support he gives is 100% up to him. I expect he has a life outside Calibre and hope he puts that first!

I'd love to see development time put into the in process server, but I realize it's not a feature that most people use, and as such should be prioritized down, especially with things like COPS in existence.

I may look into how to get COPS to play nice with the web browser in my Kobo Aura HD. (Currently, that browser will not download from COPS when the download button is clicked, but downloading the same file / format from the Calibre server works fine)
DouglasK is offline   Reply With Quote
Old 03-06-2014, 08:05 PM   #8
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 8,863
Karma: 12755553
Join Date: Feb 2009
Location: North Carolina
Device: Nexus 7
My apologies for coming to the party late.

I'm glad Kovid clarified his position.
DoctorOhh is offline   Reply With Quote
Old 03-06-2014, 09:58 PM   #9
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,154
Karma: 5381911
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
@DouglasK: Dont be fooled by top. The resource requirements for the in process server and the reverse proxy are identical. top is not an accurate way to determine a processes resource requirements. Just try the simple expedient of summing up the memory usage numbers of all processes reported by top and see that you will get many multiples of the actual physical resources on your machine.

A bit of googling should get you more info on that.
kovidgoyal is offline   Reply With Quote
Old 03-06-2014, 10:09 PM   #10
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,154
Karma: 5381911
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
In fact, depending on the MPM you use in Apache, resource consumption could be several times higher. Since if you chose the process MPM you will get one copy of the calibre server using its own resources per worker process instead of just a single process with threads when using the reverse proxy.
kovidgoyal is offline   Reply With Quote
Old 04-29-2014, 10:30 PM   #11
Ekkehard.Blanz
Junior Member
Ekkehard.Blanz began at the beginning.
 
Posts: 1
Karma: 10
Join Date: Apr 2014
Device: iPad,Kindle fire
Kovid, I can easily see your point. However, you are describing a different use case, i.e. lots of and frequent requests. @DouglasK and I have a different use case: very rare requests, almost guaranteed not to be simultaneous, where it would be advantageous if the calibre server would not be loaded all the time. Therefore, I think your initial architectural decision to provide both must be applauded. I also understand that you are hesitant to support the inProcess approach - it requires the distro (not binary) version, and those are notoriously behind the latest revision. At any rate, apache 2.4 broke both approaches the way they are described in the docs. While I had no luck with the inProcess version, I could get the reverse Proxy to work. Here is how (YMMV):
LoadModule proxy_module /usr/lib/apache2/modules/mod_proxy.so
LoadModule proxy_http_module /usr/lib/apache2/modules/mod_proxy_http.so

ProxyPass /calibre http://localhost:8080/calibre
ProxyPassReverse /calibre http://localhost:8080/calibre

SetEnv force-proxy-request-1.0 1
SetEnv proxy-nokeepalive 1
Ekkehard.Blanz is offline   Reply With Quote
Old 04-29-2014, 10:36 PM   #12
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,154
Karma: 5381911
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
If you want to cater to a use case where the calibre server is only needed very rarely, put it behind a proxy that starts it on demand and shuts it down after a period of inactivity.
kovidgoyal is offline   Reply With Quote
Old 04-30-2014, 01:45 AM   #13
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,154
Karma: 5381911
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Actually, I dont see the point of an on-demand server. A running calibre server uses no resources and should get swapped out if no requests arrive for a while and the computer is under memory pressure. So shutting it down is pretty pointless. Not to mention that I doubt Apache shuts itself down when there is no demand either. So the resource consumption would be no different in either scenario.
kovidgoyal is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Calibre Content Server and Windows Home Server 2011 HughMcC Calibre 8 10-01-2014 07:26 AM
Porting Calibre's built-in web server to a remote server? perryja Related Tools 6 05-02-2013 09:05 AM
calibre-server: Doesn't server up newly added files kaistian Calibre 5 03-17-2013 11:54 AM
calibre-server OPDS catalog - manual move to web server HaakonME Related Tools 5 09-21-2012 03:11 AM
500 Internal Server Error accessing content server Calibre 0.8.8 DaddyO57 Calibre 1 07-20-2012 06:08 PM


All times are GMT -4. The time now is 09:28 AM.


MobileRead.com is a privately owned, operated and funded community.