Question on setting up calibre-server on Pi3 running Arch Linux ARM

[haertig]

I am looking to set up a calibre-server instance on a Raspberry Pi3 running Arch Linux ARM. My Calibre library will be rsync'ed over to the Pi from my main Calibre setup on a different computer.

I see a recent version of Calibre is available for easy installation via pacman:

[root@rpi3b]# pacman -Si calibre
Repository      : community
Name            : calibre
Version         : 3.12.0-2
Description     : Ebook management application
Architecture    : aarch64
URL             : https://calibre-ebook.com/
Licenses        : GPL3
Groups          : None
Provides        : None
Depends On      : python2-six  python2-dateutil  python2-cssutils  python2-dukpy  python2-mechanize  podofo  libwmf  chmlib  python2-lxml  libusbx  python2-html5-parser  python2-pillow  shared-mime-info  python2-dnspython
                  python2-msgpack  python2-pyqt5  python2-psutil  icu  libmtp  python2-dbus  python2-netifaces  python2-cssselect  python2-apsw  qt5-webkit  qt5-svg  python2-regex  python2-pygments  mtdev  python2-unrardll
                  desktop-file-utils  gtk-update-icon-cache  optipng  udisks2
Optional Deps   : ipython2: to use calibre-debug
                  poppler: required for converting pdf to html
Conflicts With  : None
Replaces        : None
Download Size   : 29.53 MiB
Installed Size  : 92.55 MiB
Packager        : Arch Linux ARM Build System <builder+c2@archlinuxarm.org>
Build Date      : Fri 17 Nov 2017 10:24:40 PM MST
Validated By    : MD5 Sum  SHA-256 Sum  Signature
[root@rpi3b]#

I am running Arch Linux so as to be as light as possible to save resources on the Pi.
Command line only. No X. After a quick look at https://manual.calibre-ebook.com/server.html and also "calibre-server --help", it looks like what I'm trying to do is pretty well documented. But the web URL mentions:

Note:
The calibre server does not need a running X server, but it does need the X libraries installed as some components it uses link against them.

calibre-server --userdb /srv/calibre/users.sqlite --manage-users

But the above output from pacman -Si does not list the X libraries as a dependency. Nor does it list sqlite. Are there other unlisted dependencies as well? Is there documentation somewhere for ALL the dependencies that I will need to install to get a working calibre-server?

Sorry if these are stupid questions. I have decades of Linux experience, but only about two hours of Arch-specific experience. Possibly am I using the wrong options for pacman to list everything I need for Calibre?

[kovidgoyal]

I doubt pacman -Si lists dependencies of dependencies. Which is the case here. For instance calirbe uses Qt, which links againt X. And it uses apsw which links against sqlite and it uses python which also links against sqlite.

[haertig]

On the Pi, I got Calibre installed, along with NGINX reverse proxy with LetsEncrypt https and client cert authentication. Library is being rsyned from my main Calibre installation triggered by systemd timer on the Pi. Looking good!

calibre-server works nicely. No memory problems with my 400-ish book library and Pi's 1Gb system memory. Slight slowness when drawing main page (main calibre-server on powerful machine does this instantly, the Pi takes about 3 seconds, but this is acceptable). On individual book pages, the Pi-based calibre-server does not display the book cover art, but my main caliber-server does. Need to troubleshoot this minor issue. But overall, I'm happy!

Now trying to set up user profiles on Pi calibre-server. Running into an issue here. Before attempting user admin, I ran "pacman -S sqlite", so that's now installed. But I don't know sqlite (yet), so I haven't touched it other than running the pacman installation. Possibly I need to do some sqlite configuration/setup before I try to use calibre-server to manage users? See errors I'm getting below.

Manually created directory /srv/calibre, ownership by userid "calibre", the same userid I'm using to run calibre-server.

Thanks again for all the help you've given me for this project, both in this thread and in other threads I've created. I've just about got this thing nailed down to exactly what I want!

[calibre@rpi3b ~]$ calibre-server --userdb /srv/caliber/users-sqlite --manage-users

1) Add a new user
2) Edit an existing user
3) Remove a user
4) Cancel

What do you want to do? [1-4]: 1
Enter the username: guest
Traceback (most recent call last):
  File "/usr/bin/calibre-server", line 20, in <module>
    sys.exit(main())
  File "/usr/lib/calibre/calibre/srv/standalone.py", line 176, in main
    manage_users_cli(opts.userdb)
  File "/usr/lib/calibre/calibre/srv/manage_users_cli.py", line 218, in manage_users_cli
    actions[0]()
  File "/usr/lib/calibre/calibre/srv/manage_users_cli.py", line 214, in toplevel
    _('Cancel')])]()
  File "/usr/lib/calibre/calibre/srv/manage_users_cli.py", line 93, in add_user
    username = get_valid(_('Enter the username'), m.validate_username)
  File "/usr/lib/calibre/calibre/srv/manage_users_cli.py", line 54, in get_valid
    fail_message = invalidq(ans)
  File "/usr/lib/calibre/calibre/srv/users.py", line 140, in validate_username
    if self.has_user(username):
  File "/usr/lib/calibre/calibre/srv/users.py", line 137, in has_user
    return self.get(username) is not None
  File "/usr/lib/calibre/calibre/srv/users.py", line 132, in get
    for pw, in self.conn.cursor().execute(
  File "/usr/lib/calibre/calibre/srv/users.py", line 79, in conn
    self._conn = apsw.Connection(self.path)
apsw.CantOpenError: CantOpenError: unable to open database file
[calibre@rpi3b ~]$

"Unable to open database file", but permissions look correct:

[root@rpi3b]# cd /srv
[root@rpi3b]# pwd
/srv
[root@rpi3b]# ls -al
total 20
drwxr-xr-x  5 root    root    4096 Dec  4 17:48 .
drwxr-xr-x 16 root    root    4096 Dec  1 16:56 ..
drwxr-xr-x  2 calibre calibre 4096 Dec  4 17:48 calibre
dr-xr-xr-x  2 root    ftp     4096 Mar 31  2017 ftp
drwxr-xr-x  2 root    root    4096 Mar 31  2017 http
[root@rpi3b]#

[haertig]

Found another problem. When I said everything was working fine above, I had not tried to actually download a book. That fails (in Firefox). I get an error message like this example:

Firefox can’t find the file at https://my.domain.com/calibre/get/AZW3/107/Calibre

So things aren't quite right yet. This error does NOT happen if I use Chromium as my web browser. I can download find from Chromium. Hmmm...

p.s. - I am cutting-n-pasting the specific book page URL between browsers, both browsers running on the same computer. From that book detail page, I'm clicking the "Download" button. Totally different results between Chromium and Firefox for this identical operation.

[kovidgoyal]

Your issues with firefox will most likley be https related. Look in the firefox console for error messages. And an unable to open database error is because you spelt calibre wrong in your command line.

[haertig]

And an unable to open database error is because you spelt calibre wrong in your command line.

Ouch. That bruised my ego.

Thanks!

[haertig]

When I try to limit a user to only one virtual library, when they attempt to access the content-server they get a 403 error "The user guest is not allowed to access any libraries on this server".

But if I allow them access to all virtual libraries, they access the content server just fine, and can manually choose the same virtual library that I was initially trying to limit them to.

I did find something suspicious when configuring the user. When I restricted them to only one VL, I typed the library name as "Novels". Note the initial capital letter. But when I later confirmed through the content server what libraries were set, it displayed "novels" (lower case initial letter). Is this a possible bug in letter-case handling? Or something else I'm doing wrong?

[calibre@rpi3b user]$ systemctl --user stop calibre-server
[calibre@rpi3b user]$ calibre-server --userdb /srv/calibre/users-sqlite --manage-users

1) Add a new user
2) Edit an existing user
3) Remove a user
4) Cancel

What do you want to do? [1-4]: 2
Existing user names:
guest
Enter the username: guest

guest has readonly access
1) Show password for guest
2) Change password for guest
3) Change read/write permission for guest
4) Change the libraries guest is allowed to access
5) Cancel

What do you want to do? [1-5]: 4

{} has the no additional per-library restrictions

1) Allow access to all libraries
2) Allow access to only specified libraries
3) Allow access to all, except specified libraries
4) Change per-library restrictions
5) Cancel

What do you want to do? [1-5]: 2
Enter a comma separated list of library names: Novels
Allowing access only to libraries: Novels

guest has readonly access
1) Show password for guest
2) Change password for guest
3) Change read/write permission for guest
4) Change the libraries guest is allowed to access
5) Cancel

What do you want to do? [1-5]: 4

guest is currently only allowed to access the libraries named: novels
{} has the no additional per-library restrictions

1) Allow access to all libraries
2) Allow access to only specified libraries
3) Allow access to all, except specified libraries
4) Change per-library restrictions
5) Cancel

What do you want to do? [1-5]: 5

1) Add a new user
2) Edit an existing user
3) Remove a user
4) Cancel

What do you want to do? [1-4]: 4
[calibre@rpi3b user]$ systemctl --user start calibre-server

Here is the Calibre unit file:

[Unit]
Description=Calibre content server
After=network.target

[Service]
Type=simple
ExecStart=/usr/bin/calibre-server --url-prefix /calibre --num-per-page=999 --port 8080 --userdb /srv/calibre/users-sqlite --enable-auth /var/opt/calibre

[Install]
WantedBy=multi-user.target

[kovidgoyal]

library names are case-insensitive. Make sure you are actuallys erving a library named novels, i.e. use calibre-server /path/to/novels

[haertig]

Ah, got it. Thanks yet again! My mistake in misunderstanding. My library is named "Calibre" (not "Novels"), and then I want to restrict the user to the VIRTUAL LIBRARY named "Novels" that is within the "Calibre" main library.

Truncated output from "calibre-server --manage-users..."

...
guest is currently only allowed to access the libraries named: calibre
guest has the following additional per-library restrictions:
calibre: vl:Novels
...

Now that part works like I want it to.

I'm down to my last two (hopefully!) problems:

1. Firefox has failures downloading books whereas Chromium does not
2. Book cover art is not show in book detail page

I will continue investigating these two...

[haertig]

Both of the above two questions appear to be browser related. I found that Chromium on one computer DOES display book cover art in the individual book page, but another instance of Chromium on a different computer does NOT. Weird.

I keep all my eBooks in both AZW3 and EPUB format. With my main Calibre (the content server on my desktop computer, not the one I'm working on here on the Pi) I can designate AZW3 as the format to use from the Content Server webpage "Download" button in the Preferences GUI. But I cannot find any way to designate this in the stand-along calibre-server (no GUI) on the Pi. The calibre-server on the Pi defaults to EPUB for the Download button. Is there a way to default this to AZW3 without having access to the GUI on the Pi?

[kovidgoyal]

Copy over the relevant config file. IIRC it is prefs.py or simpler is to just copy the entire config dir which you get from Preferences->Miscellaneous

And on your server you can point to the config dir location via an environment variable as described here: https://manual.calibre-ebook.com/cus...ment-variables

[haertig]

Can calibre-server run as a "socket activated" service under systemd? So that it is not running continuously, but only gets started when an incoming connection attempt is made, and then exits after servicing that connection?

[kovidgoyal]

You can socket activate it, but it wont quit after servicing the connection. There is not much point in making it quit, as long as you have enabled swap on your server. If it is not used, the kernel will simply swap it out. As for how to do it, look for socket activation in --help

[haertig]

I will look more into this. Thanks. The issue I am investigating is on this Raspberry Pi with limited memory, I also want to run Nextcloud. It would be extremely rare, if ever, that calibre-server would be being accessed at the same time as Nextcloud. So in theory I should be OK running both of these applications on the same Pi, as long as neither holds on to a significant amount of memory when it's not being used.

I have a systemd timer that fires off a nightly rsync to my main Calibre setup and copies the library over to the Pi. At the end of the rsync, it restarts calibre-server on the Pi so it will pick up any changes. What I may do, is socket activate calibre-server on the Pi, then at the nightly rsync timer, simply kill calibre-server at the end of the rsync rather than restarting it. Thus calibre-server should socket-activate the next time someone tries to access it, and would pick up any changes from the rsync at that time.

This should work (I think). I'll play around with it.