Performance? calibre-server v3 on Raspberry Pi ARM

haertig · 11-27-2017, 12:45 PM

For those here who have installed this, how is the performance? My desire is to run the calibre-server on the Pi, and rsync by ebook collection from my main computer (my primary Calibre installation) to the Pi.

I understand there is no standard installation for this, but I'm OK with changing the distro on the Pi from Raspbian, compiling manually, etc. Whatever it takes. I've been a Unix/Linux command line person since the beginning of time, so "distros" don't make much difference to me.

I will also be running calibre-server through a reverse proxy using NGINX - also running on that same Pi - accessible only via HTTPS and client cert authentication.

I currently have this set up and working well on a Pi, running NIGINX on top of Raspbian Lite, but NGINX is currently directing things to my main Content Server on a different machine. I wish to put a copy of calibre-server on the Pi itself though. If performance is good enough.

I have noticed minor delays in my current setup. Only noticeable when I first access the Content Server. It takes a few seconds to draw all the book cover icons when going through reverse proxy Pi/NGINX as compared to going straight to the Content Server. Straight to the Content Server and the entire page draws instantly. My primary Content Server is running on a machine with plenty of horsepower, so this slight slowdown of icon drawing is the result of the Pi/NGINX thing. I'm wondering how MUCH slower things might get if I run a secondary Content Server directly on the Pi.

p.s., I am running my Contrent Server with --num-per-page=999 so that currently it is drawing close to 400 book covers on initial load, rather than the smaller number that is the default setting. For some reason, I loathe webpages that scroll for a bit, then say "Click here to load more". I don't know why I hate that so much, but I do. Thankfully Calibre has a way around that via --num-per-page.

kovidgoyal · 11-27-2017, 09:49 PM

I doubt CPU performance will be a big limitation. However, RAM might be. calibre's database backend is designed to load the entire db in RAM which does not play well with severly RAM limited servers.

haertig · 11-28-2017, 12:26 AM

Thanks very much for pointing me in the "look at memory usage" direction!

I did a little testing on my ~400 book library running caliber-server (this is on my larger Linux box). It's difficult to tell how much the "entire Calibre Server operation" consumes, since there are all kinds of libraries and shared stuff that the calibre-server process might set in motion.

As best I could guestimate, is that starting calibre-server alone does not take much. In my case, maybe 70k (the web browser was already running and accounted for prior to coming up with that 70k figure). Then when I hit calibre-server with a web connection, usage jumped to 300-400k, when I changed the view to detailed list, usage spiked to about 500k, then settled back down into the 400k+ range. Take all this with a grain of salt, but it's the best guesstimate that I could come up with.

So it looks like calibre-server would use about 1/2 the 1Gb total memory of the Pi. Add the OS and NGINX on top of that and the Pi would be OK I think, but without much room to spare. I'll have to do more testing on the Pi itself to check OS and NGINX memory footprints.

Bottom line - it's worth a shot to try running the caliber-server on a Pi (with my 400 book library size). I could use COPS because that's much lighter, but I really like the native Calibre Server much better. I'll probably give this Calibre setup a try under Arch Linux ARM implemented as light as I can make it, running headless. Hopefully when they finally come out with the Pi4 (sometime in 2018?) they'll give it at least 2Gb ram (and USB 3.0!)

My fallback would be to continue running calibre-server on my bigger Linux box like I'm doing now, and have NGINX on the Pi reverse proxy that. Using client certs with NGINX for security, and a user profile on Calibre to allow me to prevent write access. This is what I'm doing now (currently in test/debug phase - but it works fine). I'm not too keen on exposing my main Caliber library like this, even with write access turned off, but it is backed up daily to a separate backup server and I can restore to any day's snapshot that I want. So a rogue user that broke through Calibre's defenses would not devastate me. A full restore of my Calibre library would probably only take a few minutes. My Calibre data directory is on 1.1Gb total at this point. That's high for only 400 books, but I keep both an AZW3 and EPUB copy of each book - 1) to make sure conversion worked, and 2) for additional redundancy.

11-27-2017, 12:45 PM	#1
haertig Wizard Posts: 1,891 Karma: 31522252 Join Date: Sep 2017 Device: PW3, Fire HD8 Gen7, Moto G7, Sansa Clip v2, Ruizu X26	Performance? calibre-server v3 on Raspberry Pi ARM For those here who have installed this, how is the performance? My desire is to run the calibre-server on the Pi, and rsync by ebook collection from my main computer (my primary Calibre installation) to the Pi. I understand there is no standard installation for this, but I'm OK with changing the distro on the Pi from Raspbian, compiling manually, etc. Whatever it takes. I've been a Unix/Linux command line person since the beginning of time, so "distros" don't make much difference to me. I will also be running calibre-server through a reverse proxy using NGINX - also running on that same Pi - accessible only via HTTPS and client cert authentication. I currently have this set up and working well on a Pi, running NIGINX on top of Raspbian Lite, but NGINX is currently directing things to my main Content Server on a different machine. I wish to put a copy of calibre-server on the Pi itself though. If performance is good enough. I have noticed minor delays in my current setup. Only noticeable when I first access the Content Server. It takes a few seconds to draw all the book cover icons when going through reverse proxy Pi/NGINX as compared to going straight to the Content Server. Straight to the Content Server and the entire page draws instantly. My primary Content Server is running on a machine with plenty of horsepower, so this slight slowdown of icon drawing is the result of the Pi/NGINX thing. I'm wondering how MUCH slower things might get if I run a secondary Content Server directly on the Pi. p.s., I am running my Contrent Server with --num-per-page=999 so that currently it is drawing close to 400 book covers on initial load, rather than the smaller number that is the default setting. For some reason, I loathe webpages that scroll for a bit, then say "Click here to load more". I don't know why I hate that so much, but I do. Thankfully Calibre has a way around that via --num-per-page.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Calibre on Raspberry Pi 3	pckopp	Devices	2	12-21-2016 12:19 PM
Content server on Raspberry Pi. Move database error	aerospyke	Library Management	5	06-02-2016 09:42 AM
Content Server on Raspberry Pi not working with password	darryl	Related Tools	10	02-03-2016 10:22 AM
Calibre on Raspberry Pi	ottischwenk	Related Tools	8	03-01-2013 09:43 PM

11-27-2017, 09:49 PM	#2
kovidgoyal creator of calibre Posts: 45,246 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I doubt CPU performance will be a big limitation. However, RAM might be. calibre's database backend is designed to load the entire db in RAM which does not play well with severly RAM limited servers.

11-28-2017, 12:26 AM	#3
haertig Wizard Posts: 1,891 Karma: 31522252 Join Date: Sep 2017 Device: PW3, Fire HD8 Gen7, Moto G7, Sansa Clip v2, Ruizu X26	Thanks very much for pointing me in the "look at memory usage" direction! I did a little testing on my ~400 book library running caliber-server (this is on my larger Linux box). It's difficult to tell how much the "entire Calibre Server operation" consumes, since there are all kinds of libraries and shared stuff that the calibre-server process might set in motion. As best I could guestimate, is that starting calibre-server alone does not take much. In my case, maybe 70k (the web browser was already running and accounted for prior to coming up with that 70k figure). Then when I hit calibre-server with a web connection, usage jumped to 300-400k, when I changed the view to detailed list, usage spiked to about 500k, then settled back down into the 400k+ range. Take all this with a grain of salt, but it's the best guesstimate that I could come up with. So it looks like calibre-server would use about 1/2 the 1Gb total memory of the Pi. Add the OS and NGINX on top of that and the Pi would be OK I think, but without much room to spare. I'll have to do more testing on the Pi itself to check OS and NGINX memory footprints. Bottom line - it's worth a shot to try running the caliber-server on a Pi (with my 400 book library size). I could use COPS because that's much lighter, but I really like the native Calibre Server much better. I'll probably give this Calibre setup a try under Arch Linux ARM implemented as light as I can make it, running headless. Hopefully when they finally come out with the Pi4 (sometime in 2018?) they'll give it at least 2Gb ram (and USB 3.0!) My fallback would be to continue running calibre-server on my bigger Linux box like I'm doing now, and have NGINX on the Pi reverse proxy that. Using client certs with NGINX for security, and a user profile on Calibre to allow me to prevent write access. This is what I'm doing now (currently in test/debug phase - but it works fine). I'm not too keen on exposing my main Caliber library like this, even with write access turned off, but it is backed up daily to a separate backup server and I can restore to any day's snapshot that I want. So a rogue user that broke through Calibre's defenses would not devastate me. A full restore of my Calibre library would probably only take a few minutes. My Calibre data directory is on 1.1Gb total at this point. That's high for only 400 books, but I keep both an AZW3 and EPUB copy of each book - 1) to make sure conversion worked, and 2) for additional redundancy.

Advert