Calibre-Web and Calibre side by side?

[coleman]

I've been migrating my installations over to Linux systems recently, and I've initially installed Calibre-Web using one of the Docker images. That seems pretty good, but there's a couple spots I'm fuzzy on:

1. Do you also install Calibre typically, to handle whatever things Calibre-Web doesn't?
2. If I install them both, do I need to be careful not to run them at the same time (mostly concerned about simultaneous access to the database).
3. I think I need to install Calibre if for no other reason then to get the converter, as the image I used (linuxserver/calibre-web) doesn't seem to have it included.

I think that's all my questions off the top of my head.
Thanks!

[eschwartz]

This seems to be completely unrelated software from https://github.com/janeczku/calibre-web -- as far as I can tell, it treats your calibre library as completely read-only, so it should make absolutely no difference whether calibre is running at the same time, as "calibre-web" will never try to modify the database.

That being said, if you're running server software anyway, I'd just go with the builtin server in calibre. You'll need calibre installed anyway in order to create the database, unless you're syncing your desktop library to a remote cloud instance.

[haertig]

I run Calibre on my desktop computer. I rsync the Calibre database and files over to my server (so the server has a second copy, distinct from my primary database). I run calibre-web in a Docker container on my server, with the servers copy of the Calibre database mounted read-only into the container. eschwartz mentions that calibre-web only accesses the Calibre data in read-only mode, which is probably true, but I don't recall seeing that mentioned in the calibre-web documentation. However, if you also mount that data read-only into the Docker container, you guaranty that it is read-only.

Just a few minutes ago, coincidentally, I posted my Docker config for calibre-web in a different thread. Here's a link to that:

https://www.mobileread.com/forums/sh...1&postcount=66

[coleman]

Originally Posted by eschwartz

This seems to be completely unrelated software from https://github.com/janeczku/calibre-web -- as far as I can tell, it treats your calibre library as completely read-only, so it should make absolutely no difference whether calibre is running at the same time, as "calibre-web" will never try to modify the database.

I suppose it depends on how you are defining unrelated, I put the thread in related tools because it does depend on the Calibre database. I don't think it's read only though, you can upload content through the web interface. I'm using it partly because it does have a nicer/more complete web UI then the one built in to Calibre. It might be that I made an incorrect assumption about the read-only state, I haven't yet re-installed a calibre instance to see if the things I uploaded through the web UI will be visible in Calibre as well.

That being said, if you're running server software anyway, I'd just go with the builtin server in calibre. You'll need calibre installed anyway in order to create the database, unless you're syncing your desktop library to a remote cloud instance.

Yup, that was the part I was trying to figure out, if I can have Calibre and Calibre-web working against the same dataset easily. I already had a Calibre db, I moved my library from a Windows install. Now it became a question of how to most effectively maintain that. Calibre-web does some, but seemingly not quite all of the things I used in Calibre, so that's the problem I'm trying to solve.

Thanks!

[coleman]

Originally Posted by haertig

I run Calibre on my desktop computer. I rsync the Calibre database and files over to my server (so the server has a second copy, distinct from my primary database).

Hmm...that's an intriguing idea. I'm not keen on having 2 copies of the library, but perhaps that's the best way to mitigate some of the scenarios I'm concerned about being problematic. I suppose I can always manual rsync when I want to force calibre-web to update too. I'll chew on that idea for a bit.
Thanks!

[BetterRed]

Curious - is there such a thing as a transaction processing monitor for any of the consumer grade operating systems - Windows, Linux, MacOS. Seems to me that's what's needed if you want to simultaneously update a common database from multiple hosts.

The only ones that I know of that even come close are, IBM's TXSeries, which is a subset of CICS, and Oracle's Tuxedo I hasten to add my knowledge is very rusty - prehistoric even

BR

[haertig]

I used to maintain all of my media (eBooks, Movies, TV Shows, Audiobooks, Pictures, etc.) on my desktop computer, rsync that all over to a second server, and then provide web-based access to all this stuff from that second server. I used a one-way rsync, so that the desktop would overwrite anything on the server, but not the other way around. I also have a third server, my backup server, that does all my backups - currently for seven computers in my household and some remote computers. This backup server runs a client/server backup solution called "UrBackup". I currently have it backing up both Linux and Windows computers, some of those remotely via VPN.

Anyway, by having a second copy of everything on my server, I had isolation from getting anything corrupted on my desktop (which held my primary copies).

Lately though, I have started keeping my only copy of the media on the server (except for the backups on that third sever - those are still redundant copies of everything). So with my media primarily stored on the server now, that server provides NFS mounts to my desktop. It is from my desktop that I write and update all these server files via the NFS mounts. Additionally, I run Docker on the server and to make my media available on the web for myself and family and a few friends. So I have docker containers for Calibre-web, Airsonic, Booksonic, Plex, Piwigo, Nextcloud, etc. All of these containers get read-only access to the servers files (except for Nextcloud since that would totally defeat its purpose). So the Docker containers have read-only, and my desktop computer has read-write (via NFS). My Calibre database is the one media thing that I have not moved totally to the server yet. So as it exists today, the Calibre stuff is primarily on my desktop and then rsynced over to the server and the server works off it's redundant copy. If somehow the servers copy of Calibre gets corrupted, it will automatically be overwritten and fixed on the next nightly rsync from my desktop.

My primary motivation for moving the media from my desktop to the server was disk space. Basically, I was running myself too low on the desktop computer. My server has far more disk space available and more horsepower.
That extra horsepower is mostly wasted - the server doesn't need anywhere as much as it has available, but it's always nice to have too much rather than too little. Things like Airsonic (that are written in Java) are notorious memory hogs. Java is the pig that tries to gobble up all available resources wherever you let it loose. We went from C (very efficient), to C++ (slightly less efficient), to Java (a.k.a. C++-) which is just a terrible hog if you don't know what you're doing, and it appears many programmers don't.

[coleman]

Originally Posted by BetterRed

Curious - is there such a thing as a transaction processing monitor for any of the consumer grade operating systems - Windows, Linux, MacOS. Seems to me that's what's needed if you want to simultaneously update a common database from multiple hosts.

A commoin approach for simultaneous access these days would be to move up to a multiuser RDBMS (i.e. SQL Server, Oracle, PostGRESQL..the list goes on), and possibly adding in some sort of message queuing (MSMQ, or Service Bus in Azure, for instance). The queuing services might be some of what you are referring to as a transaction monitor, but full-blown multi-user databases fundamentally support transaction concurrency and consistency. It's non-trivial work+infrastructure to add those things though. Well, switching to a full blown database would probably be not too bad, but there's no real reason to.

The problems I'm considering probably haven't even crossed most Calibre user's minds, and wouldn't be an issue for me even if I wasn't using Calibre-web for the web piece. Honestly, the chances are that it would even be a problem for me are slim, I'm just over-cautious having lived through developing apps way back in the day backed by Access databases.

[BetterRed]

Originally Posted by coleman

A commoin approach for simultaneous access these days would be to move up to a multiuser RDBMS (i.e. SQL Server, Oracle, PostGRESQL..the list goes on), and possibly adding in some sort of message queuing (MSMQ, or Service Bus in Azure, for instance). The queuing services might be some of what you are referring to as a transaction monitor, but full-blown multi-user databases fundamentally support transaction concurrency and consistency. It's non-trivial work+infrastructure to add those things though. Well, switching to a full blown database would probably be not too bad, but there's no real reason to.

The problems I'm considering probably haven't even crossed most Calibre user's minds, and wouldn't be an issue for me even if I wasn't using Calibre-web for the web piece. Honestly, the chances are that it would even be a problem for me are slim, I'm just over-cautious having lived through developing apps way back in the day backed by Access databases.

'Requests' to be able to run two or more instances of calibre.exe, calibre-server, calibredb.exe, and 3rd party programs against the same set of libraries are quite frequent - maybe one a week. And a common cause of things going haywire is when users don't heed the warnings in the FAQ section of the manual.

Not a fan of shoehorning everything into an RDBMS (-ve experience with Sybase and SQLServer), I grew up on Bachman databases and Univac TIPS and GE whatever it was**. So, I have a bit of an anti-Codd bias.

As you say it's a non-trivial issue, hoped a silver bullet might have eventuated when i wasn't looking

Added: ** It was TDS (Transaction Driven Subsystem) - pronounced TeDiouS.

BR

[coleman]

Well, I ended up just disable uploads and editing on the calibre-web side, and then installing calibre and targeting the same folder. I think it'll be fine, I really don't have enough going on where I expect I'll run into an issue. I can always stop the docker instance running calibre-web, while I do stuff in calibre, if I get that worired about it.

Simplest solutions and all that.

[haertig]

Originally Posted by coleman

I can always stop the docker instance running calibre-web, while I do stuff in calibre, if I get that worired about it.

Just set up your Docker container like this (excerpt from docker-compose.yml). Your Calibre library will be mounted read-only into your Docker container, thus safe.

  calibre-web:
    image: linuxserver/calibre-web
    container_name: calibre-web
    volumes:
      - /var/opt/docker/calibre-web/config:/config
      - /var/opt/calibre:/books:ro
    environment:
      - PUID=1002
      - PGID=1001
    networks:
      - frontend
    restart: unless-stopped

If you're not using docker-compose, then you can do the same thing for a standalone container with docker-create (the syntax is slightly different in this case).