Best way to allow for remote access of my Calibre library?

Magic815 · 05-07-2022, 03:38 PM

My current installation:
My Calibre Library and database are located on an always-on server which is on my home network:
\\xxx.xxx.x.xxx\Data\Media\EBooks

And I interface with that library from my Windows PC, where my single installation of Calibre resides. So whenever I want to add eBook(s) to my Calibre library, I turn on my Windows PC and use its Calibre install to add the book, edit the metadata, run the various plugins (Count Pages, etc.), convert to my output format of choice, and then send the eBook(s) to my device by connecting it to my Windows PC via USB.

What I'd like to do:
I'd like to be able to set up 24/7 remote access to my eBook library, so that I can also have the option to add a book to my eReader while not at home (when I don't have USB connection/access to my Windows PC).

Am I understanding that one option would be to turn on Calibre Content Server from my Windows PC Calibre install, reverse proxy to a domain I run, and that would provide me with website remote access to my library? But I'm assuming the issue is now my Windows PC needs to be on 24/7, to provide this 24/7 remote access? (And does Calibre need to be open/running on the Windows PC to keep the Content Server link going?)

Are there other avenues to allow for remote access to my Calibre library that don't require my Windows PC to now be always on? The always-on server (xxx.xxx.x.xxx) on my home network (where the library/dB resides anyways) is able to run docker containers, which would always be running. Would it make sense for me to consider running Calibre, calibre-web, or COPS as a docker container on my server - in addition to my Windows PC Calibre install? Or would there be issues with two things (my server docker and my Windows PC Calibre install) accessing the same library/dB? If that would be an issue, could there be a way to have a synced duplicate of my library/dB that the server docker container points to, while my Windows PC install sticks to interfacing with the main library/dB location? Has anyone got suggestions on how to go about this?

Magic815 · 05-07-2022, 05:40 PM

I can't seem to edit my above post, but I realized my current installation has a critical flaw. It has the dB file across the network, and would be prone to corruption over time. So what I've done is re-done my library and dB to be back on my local Windows PC where Calibre is installed.

With that corrected setup, would I be able to use something like rsync to make a one-way copy of the library and dB to my server - which I could run something like calibre-web against? Has anyone done something like that?

PeterT · 05-07-2022, 07:25 PM

Yep. I use FreeFileSync to mirror my calibre library to a QNAP server (monitoring for changes to calibre's database and waiting 60 seconds). The QNAP in turn shares via NFS to a Raspberry Pi with multiple docker images, including LinuxServer's SWAG (an NGINX server) and a couple of calibre-web instances. The SWAG container handles redirection to the calibre-web containers allowing me consistent access by name from both internal and external devices.

Sent from my Pixel 4a using Tapatalk

Magic815 · 05-07-2022, 07:27 PM

Quote:

Originally Posted by PeterT

Yep. I use FreeFileSync to mirror my calibre library to a QNAP server (monitoring for changes to calibre's database and waiting 60 seconds). The QNAP in turn shares via NFS to a Raspberry Pi with multiple docker images, including LinuxServer's SWAG (an NGINX server) and a couple of calibre-web instances. The SWAG container handles redirection to the calibre-web containers allowing me consistent access by name from both internal and external devices.

Sent from my Pixel 4a using Tapatalk

Awesome. I actually have FreeFileSync already on my Windows PC, but have only ever used it for manually triggered sync'ing. Would you mind walking me through the details of how you set up a trigger based on a change to calibre's database (with a 60 second wait)?

PeterT · 05-07-2022, 10:36 PM

Quote:

Originally Posted by Magic815

Awesome. I actually have FreeFileSync already on my Windows PC, but have only ever used it for manually triggered sync'ing. Would you mind walking me through the details of how you set up a trigger based on a change to calibre's database (with a 60 second wait)?

I'm not at the PC right now but I seem to recall it being done by RealTimeSync. See https://freefilesync.org/manual.php?topic=realtimesync

Quote:

RealTimeSync receives change notifications directly from the operating system in order to avoid the overhead of repeatedly polling for changes. Each time a file or folder is created/updated/deleted in the monitored directories or their sub directories, RealTimeSync waits until a user-configurable idle time has passed in which no further changes were detected, and then runs the command line. This makes sure the monitored folders are not in heavy use when starting a synchronization.

Sent from my Pixel 4a using Tapatalk

Magic815 · 05-07-2022, 10:40 PM

Quote:

Originally Posted by PeterT

I'm not at the PC right now but I seem to recall it being done by RealTimeSync. See https://freefilesync.org/manual.php?topic=realtimesync
Sent from my Pixel 4a using Tapatalk

Cool, that's what I figured as I dug into it more. I've got a realtimesync batch script that starts running in the background when I turn my Windows PC on. Overall, it seems like the duplicate/sync'd folder on my server is working, but I am noticing a build-up of ".fuse_hidden" files. I believe these are remnants of the dB file that is trying to get deleted by realtimesync, but calibre-web is still "using them". Were you able to do anything to clear those .fuse_hidden files?

PeterT · 05-10-2022, 09:41 AM

Weird. I've never seen those files appear.

Sent from my Pixel 4a using Tapatalk

DNSB · 05-10-2022, 12:50 PM

I've seen .fuse_hidden files under Ubuntu when I deleted files that were still being used by something. In my case, it was a Samba bug that was causing the issue by not releasing files that had been accessed over the network.

Can you stop calibre-web before running the sync?

They did clear up after a reboot.

Magic815 · 05-10-2022, 02:00 PM

In my case, it's definitely calibre-web using them. A quick restart of the calibre-web docker container, and the .fuse_hidden file immediately goes away. I guess I'll have to just set up a cron job to rest the docker a few times a day? It's just weird that others haven't had that issue.

PeterT · 05-10-2022, 02:41 PM

I have a sense it might be related to how our different environments run.

I go from Windows to QNAP via SMB, then have an NFS mount to a raspberry Pi with docker containers.

On the QNAP I have a python script running that when it detects a change in the calibre database sends a command to calibre-web to reload the database.

Sent from my Pixel 4a using Tapatalk

05-07-2022, 03:38 PM	#1
Magic815 Enthusiast Posts: 34 Karma: 98 Join Date: May 2022 Device: Kindle Paperwhite 11th Gen (2021)	Best way to allow for remote access of my Calibre library? My current installation: My Calibre Library and database are located on an always-on server which is on my home network: \\xxx.xxx.x.xxx\Data\Media\EBooks And I interface with that library from my Windows PC, where my single installation of Calibre resides. So whenever I want to add eBook(s) to my Calibre library, I turn on my Windows PC and use its Calibre install to add the book, edit the metadata, run the various plugins (Count Pages, etc.), convert to my output format of choice, and then send the eBook(s) to my device by connecting it to my Windows PC via USB. What I'd like to do: I'd like to be able to set up 24/7 remote access to my eBook library, so that I can also have the option to add a book to my eReader while not at home (when I don't have USB connection/access to my Windows PC). Am I understanding that one option would be to turn on Calibre Content Server from my Windows PC Calibre install, reverse proxy to a domain I run, and that would provide me with website remote access to my library? But I'm assuming the issue is now my Windows PC needs to be on 24/7, to provide this 24/7 remote access? (And does Calibre need to be open/running on the Windows PC to keep the Content Server link going?) Are there other avenues to allow for remote access to my Calibre library that don't require my Windows PC to now be always on? The always-on server (xxx.xxx.x.xxx) on my home network (where the library/dB resides anyways) is able to run docker containers, which would always be running. Would it make sense for me to consider running Calibre, calibre-web, or COPS as a docker container on my server - in addition to my Windows PC Calibre install? Or would there be issues with two things (my server docker and my Windows PC Calibre install) accessing the same library/dB? If that would be an issue, could there be a way to have a synced duplicate of my library/dB that the server docker container points to, while my Windows PC install sticks to interfacing with the main library/dB location? Has anyone got suggestions on how to go about this?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Remote access to Calibre library, WITHOUT content server?	haertig	Calibre	1	10-18-2017 11:24 PM
Another remote access question	jjansen	Calibre Companion	11	03-18-2017 07:12 AM
Remote access to calibre library	NSILMike	Calibre Companion	2	08-21-2014 09:28 AM
Use calibre to manage remote calibre library	aarcane	Related Tools	8	04-06-2013 01:40 AM
Dish Remote Access	ptsenter	enTourage eDGe	1	09-27-2011 08:47 AM

05-07-2022, 05:40 PM	#2
Magic815 Enthusiast Posts: 34 Karma: 98 Join Date: May 2022 Device: Kindle Paperwhite 11th Gen (2021)	I can't seem to edit my above post, but I realized my current installation has a critical flaw. It has the dB file across the network, and would be prone to corruption over time. So what I've done is re-done my library and dB to be back on my local Windows PC where Calibre is installed. With that corrected setup, would I be able to use something like rsync to make a one-way copy of the library and dB to my server - which I could run something like calibre-web against? Has anyone done something like that?

05-07-2022, 07:25 PM	#3
PeterT Grand Sorcerer Posts: 12,166 Karma: 73448616 Join Date: Nov 2007 Location: Toronto Device: Nexus 7, Clara, Touch, Tolino EPOS	Yep. I use FreeFileSync to mirror my calibre library to a QNAP server (monitoring for changes to calibre's database and waiting 60 seconds). The QNAP in turn shares via NFS to a Raspberry Pi with multiple docker images, including LinuxServer's SWAG (an NGINX server) and a couple of calibre-web instances. The SWAG container handles redirection to the calibre-web containers allowing me consistent access by name from both internal and external devices. Sent from my Pixel 4a using Tapatalk

05-10-2022, 09:41 AM	#7
PeterT Grand Sorcerer Posts: 12,166 Karma: 73448616 Join Date: Nov 2007 Location: Toronto Device: Nexus 7, Clara, Touch, Tolino EPOS	Weird. I've never seen those files appear. Sent from my Pixel 4a using Tapatalk

05-10-2022, 12:50 PM	#8
DNSB Bibliophagist Posts: 35,401 Karma: 145435140 Join Date: Jul 2010 Location: Vancouver Device: Kobo Sage, Forma, Clara HD, Lenovo M8 FHD, Paperwhite 4, Tolino epos	I've seen .fuse_hidden files under Ubuntu when I deleted files that were still being used by something. In my case, it was a Samba bug that was causing the issue by not releasing files that had been accessed over the network. Can you stop calibre-web before running the sync? They did clear up after a reboot.

05-10-2022, 02:00 PM	#9
Magic815 Enthusiast Posts: 34 Karma: 98 Join Date: May 2022 Device: Kindle Paperwhite 11th Gen (2021)	In my case, it's definitely calibre-web using them. A quick restart of the calibre-web docker container, and the .fuse_hidden file immediately goes away. I guess I'll have to just set up a cron job to rest the docker a few times a day? It's just weird that others haven't had that issue.

05-10-2022, 02:41 PM	#10
PeterT Grand Sorcerer Posts: 12,166 Karma: 73448616 Join Date: Nov 2007 Location: Toronto Device: Nexus 7, Clara, Touch, Tolino EPOS	I have a sense it might be related to how our different environments run. I go from Windows to QNAP via SMB, then have an NFS mount to a raspberry Pi with docker containers. On the QNAP I have a python script running that when it detects a change in the calibre database sends a command to calibre-web to reload the database. Sent from my Pixel 4a using Tapatalk