Calibre-server not updating

[skells]

I am keeping my library in sync using Dropbox, but I prefer to use my server to host the calibre web site rather than drobpox itself. I am using calibre-server on Fedora 14 and it works great except that the site is not updating when I add new content. If I kill the process and relaunch, then it's up to date, but it's far from ideal. I am using the --with-library /path --daemonize --auto-reload switches in that order. Is there a better way?

Also, I'm not so great at init scripts, does anyone know of how to launch calibre-server as an init daemon on fedora/red hat?

Thanks

[kovidgoyal]

The calibre server uses the last modified timestamp of the metadata.db file to check if it needs to refresh itself. Presumably, that timestamp is not being changed in tour setup for some reason.

[itimpi]

If any process is keeping the metadata.db file open, then that will stop the Dropbox synchronization from taking place (including updating file timestamp). Sounds to me as if that could be the problem.

[skells]

I only keep the full program open when necessary and when I close it I see that the metadata.db gets synced on dropbox afterward. Calibre-server is read-only on the database, right? I'm not sure what else could be keeping it open but I'll check the machines I replicate with.

[chaley]

Originally Posted by skells

Calibre-server is read-only on the database, right?

No. It is read/write, because that is the interface SQLite presents to python. It doesn't write, but it can.

[skells]

That seems kind of counter-productive. If nothing else is keeping a lock on the database and I can verify that time stamps are synced to the machine running calibre-server, how can I set it to automatically refresh itself? Am I using the wrong arguments?

[chaley]

Originally Posted by skells

That seems kind of counter-productive. If nothing else is keeping a lock on the database and I can verify that time stamps are synced to the machine running calibre-server, how can I set it to automatically refresh itself? Am I using the wrong arguments?

We don't have a choice. The python sqlite interface opens the DB read/write.

As for the rest of your question: are you saying that dropbox is changing the timestamp of the file and calibre-server is not seeing that change?

If dropbox is not updating the database while calibre-server is running, then I see only two workarounds:
1) periodically stop and restart calibre-server
2) use one of the other techniques to serve your content. For example, it might be easier to start/stop the server using the WSGI interface. You could use calibre2opds to periodically generate a new website. You could also use the calibre php server. All of these options bring along some inconveniences, so I would suggest option 1 as the best way forward.

[DoctorOhh]

Originally Posted by chaley

As for the rest of your question: are you saying that dropbox is changing the timestamp of the file and calibre-server is not seeing that change?

If dropbox is not updating the database while calibre-server is running, then I see only two workarounds:

Dropbox no longer locks the metadata.db file when calibre is open. Any changes to metadata.db are quickly passed through to dropbox and the timestamp is changed. It has been this way since late in the last beta phase. The OP should ensure he is using the latest dropbox, version 1.0.10. New versions automatically update, but older versions didn't.

I have not verified that dropbox works this way for the content server. I would be surprised if it operated differently, but then again...

[chaley]

Originally Posted by dwanthny

Dropbox no longer locks the metadata.db file when calibre is open. Any changes to metadata.db are quickly passed through to dropbox and the timestamp is changed.

Thanks for the info!

You seem to be describing sending data to the cloud for open files. What about the other way? Will dropbox modify the db using data from the cloud when calibre has the db open?

[DoctorOhh]

Originally Posted by chaley

You seem to be describing sending data to the cloud for open files. What about the other way? Will dropbox modify the db using data from the cloud when calibre has the db open?

No. If I have dropbox open on my machine and someone on another machine decides to add a book(which is a no-no) my computer will never see that book. I might get a file called metadata(Home-machine's conflicted copy 2010-12-27).db file on my machine. This tells me the machine that created the file and the date it was created/updated.

Since I am the only person who updates things it isn't a problem. But last week I did leave calibre open on my home machine and updated many files on my work machine. I restarted calibre but certain books were missing or covers were missing. Because the database file structure entries were updated but the metadata.db file wasn't.

Upon realizing what I had done I closed calibre then went into the calibre library and copied the conflicted metadata.db file to metadata.db, restarted calibre and all of my changes were there. I could also have gone to the web and restored the last updated version that was timestamped while I was at work.

Using dropbox you do have to pay attention when you have apparent errors or the fix might take a lot longer then it should. In the above case it took just a couple of minutes. If I didn't realize the nature of the problem I could easily have cursed out calibre (when in fact it would have been human error) and entered everything again.

So far restoring books from my dropbox as protected me from myself more then once.

[skells]

Yep, I'm running the latest and greatest of Calibre and dropbox. Dropbox definitely passes the timestamp change. Ah well, at this point I just have a cron job stop and restart calibre-server.

[Dovahn]

I'm having the same problem. I'm using calibre-server --with-library /path --daemonize --auto-reload and syncing the server library using SSH ftp (not dropbox, so the dropbox issues don't apply to me). I can confirm that the last modified timestamp is being updated on the server, but the running calibre-server process doesn't seem to be updating.

Are other similar setups working properly? Does the server check for updates on a fixed schedule? I can't seem to get the server to update while its running. Thanks for your help.

[Tedderouni]

Has anyone gotten this working? I'm having the same problem. I'm not using Dropbox in my setup, I just have Calibre-server 0.7.52 running on a Fedora 14 server. I can't get the server to update its content without restarting it after adding new ebooks to the library. The metadata.db timestamp is being updated, and lsof confirms that the only process holding it open is calibre-server. My startup command is calibre-server --with-library /path -p 80 --daemonize --auto-reload. Is there a way to make it update automatically without a restart when new content is added to the library? Or at least is there a way to manually make it re-read metadata.db so I can avoid a total restart?

[skells]

I ended up running calibre-server with an init script then I schedule a periodic restart of the service. Not optimal, but it works. I'm just not sure what the --auto-reload is for, it doesn't seem to do anything.

[Tedderouni]

I seem to have solved the problem on my end. I'm running this on a virtual machine, and I noticed the system time was off by a couple minutes. So I enabled ntpd, and once it had corrected the time, calibre-server magically started updating automatically. I'm guessing this is because I have my calibre library shared via NFS to multiple computers, and since their time wasn't synced to the VM's time, it broke calibre's timestamp checking of metadata.db. To start ntpd, type 'service ntpd start' and to enable it at boot, type 'chkconfig ntpd on'. If it's not installed, type 'yum install ntp' to install it.

Also, if you're still looking for an init script, I've come up with one. As root, paste the script below into a file named calibre in /etc/init.d/ and be sure to at least change the path to your library and anything else you need to do differently in the INI line (line 14). Then type 'chkconfig --add calibre' to enable it and 'service calibre start' to start it. The script is fairly generic, but I did add a workaround for calibre's command line segfaults, and also added a check to see if it's already running, since the built-in checking didn't seem to work for this for some reason.

Note that this script sets the port to 80, the default HTTP port. Specifically, this is done by the "-p 80" in the INI line. With this setting, you can go to the web interface using only the address, without having to specify a port, i.e. http://your.calibre.address/. However, if you already have another web server running on port 80 on the same interface, calibre will have to use a different port. Calibre server defaults to port 8080, so if you remove the "-p 80" you'll have to use http://your.calibre.address:8080/. Or if you want to use a different port, specify it with "-p <port>" and then use http://your.calibre.address:<port>/.

#!/bin/sh
#
#
# chkconfig: 345 99 15
# description:   Start/Stop the Calibre Server.
# processname: calibre-server
# pidfile: /var/run/calibre.pid

# Source function library.
. /etc/init.d/functions

#Edit the variables to suit your file locations
EXE="/usr/bin/calibre-server"
INI="--with-library /path/to/calibre_library/ -p 80 --daemonize --auto-reload --pidfile /var/run/calibre.pid"
PID="/var/run/calibre.pid"

RETVAL=0

# See how we were called.

start() {
        echo -n "Starting Calibre Server: "

	# See if it's already running
	RUNNING=0
	ps -ef |grep -v grep |grep -q calibre-server
	[ $? -eq 0 ] && RUNNING=1

	# The ps in the daemon command makes the status OK despite Calibre's expected segfault:
        [ $RUNNING -eq 0 ] && \
		daemon "${EXE} ${INI} 2> /dev/null ; ps -ef |grep -v grep |grep -q \"calibre-server ${INI}\"" || \
		echo "Calibre server already running."

        RETVAL=$?
        echo
        [ $RETVAL -eq 0 ] && touch ${PID}
        return $RETVAL
}

stop() {
        echo -n "Stopping Calibre Server: "
        killproc ${EXE}
        RETVAL=$?
        echo
        [ $RETVAL -eq 0 ] && rm -f ${PID}
        return $RETVAL
}

rhstatus() {
        status ${EXE}
}

restart() {
        stop
        start
}

case "$1" in
  start)
        start
        ;;
  stop)
        stop
        ;;
  restart)
        restart
        ;;
  status)
        rhstatus
        ;;
  *)
        echo "Usage: ${EXE} {start|stop|status|restart}"
        exit 1
esac

exit $?