[Adoby]

I suspect there are many that use rsync in weird and wonderful ways to backup their calibre libraries.

There are two reasons why it is nice to use rsync to make snapshots.

The first reason is that it syncs. So only modified files have to be copied. This means that it is very fast and efficient.

The second reason is that rsync can create hard links to unchanged files, so each snapshot still looks like a full backup, but unchanged files are only copied once. This means that you can have a very large amount of snapshots, and they don't take up much more space than they really need. No need to store duplicate files. Each file only has to be copied if it changes. The hard links are to previous copies in the snapshots. And these hard linked files use a copy counter, so you can freely delete snapshots. Only when you delete the last copy is that file really deleted.

Here is how I do it. (Ubuntu 14.04.01)

~/bin/snapshot_calibre.sh
This executable script is a general "snapshotter" that I have written and configured to make snapshots of my calibre libraries. It has to be edited to specify where the calibre libraries are and where the snapshots are to be stored, and for how many days you want to keep snapshots.

This script can easily be copied and changed to make snapshots of other folders as well. I also use it to make snapshots of my documents folder. It is then set up to make a new snapshot every time I start the computer.

It is best if you have the snapshot on an other drive, or even on another computer. I have all my snapshots on my NAS.

If you need to use a snapshot to restore lost data, copy the entire snapshot before you start changing or editing anything. Otherwise you may mess up other snapshots.

~/bin/calibre
This is also a script that is executable. All it does is launch /opt/calibre/calibre and then when calibre is exited, it immediately runs ~/bin/snapshot_calibre.sh. The nice thing about this is that commands in ~/bin takes precedence over other commands. So there is no need to change anything else to automate the snapshots. The normal launcher in Unity will use this script and updates of calibre will not overwrite anything. I haven't tested this trick in other environments...

One cause of problem may be if you start and exit calibre quickly several times, before snapshot_calibre.sh has had time to sync. Especially the first time you sync, when the sync can take a while. Just don't do it.


~/bin/snapshot_calibre.sh

Spoiler:

#!/bin/bash
# General folder snapshot maker. Makes snapshots of calibre libraries.
# Makes use of rsync and hardlinks to create snapshots that look like full backups.
# Don't use or update files in a snapshot. Copy the snapshot first.
# Deletes snapshots older than a certain number of days. (30)

snapshot_path='/path to where you keep your calibre library snapshots'
source_path='/path to where you keep your calibre library'
age_snapshot='30'
snapshot_name=$(date +%F_%T)

# Delete earlier failed attempt at making a snapshot, if any
rm -rf "$snapshot_path"/unfinished

# Find latest snapshot, if any
latest_snapshot=$(ls -rt "$snapshot_path" | tail -1)

# Create folder for this snapshot, unfinished so far...
mkdir "$snapshot_path/unfinished"

# Grab new snapshot. Use rsync to create hard links to files already in latest snapshot(s).
rsync -a --delete \
--link-dest="$snapshot_path"/"$latest_snapshot" \
"$source_path" \
"$snapshot_path"/unfinished

# Finished!
if [ $? -eq 0 ]; then
mv "$snapshot_path"/unfinished "$snapshot_path"/"$snapshot_name"
else
echo "Snapshot failed. Check free space and try again."
fi

# Delete snapshots older than $age_snapshot
find $snapshot_path -maxdepth 1 -type d -mtime +$age_snapshot -print0 | xargs -0 rm -rf

~/bin/calibre

Spoiler:

#!/bin/bash
# calibre launcher that takes a snapshot of the calibre libraries.
# Call it just "calibre" and place in ~/bin and make executable
# And make sure snapshot_calibre.sh is set up correctly.
/opt/calibre/calibre
snapshot_calibre.sh

How do you use rsync?