Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 01-19-2020, 01:01 AM   #1
qwertyuio
Junior Member
qwertyuio began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Jan 2019
Location: Australia
Device: BOOX Nova Pro
Backing up metadata.db on exit

After messing up a bulk edit I am wondering if it is possible to get calibre to create a backup copy of metadata.db each time it is shutdown. That is, on exit, copy metadata.db to metadata-backup-yymmdd.db (or some other numbering system) up to a maximum number of backups.

This would allow me to restore a previous metadata.db and run check_library to revert to a previous state. I do keep regular backups of my filesystem but these are only run once a day. When I spend an afternoon cleaning up metadata I usually start and stop calibre several times so there would be the potential to save a lot of work reverting to a known good state.

I thought of wrapping calibre in a script to do this but I have several libraries so it is a bit more complicated than a 3 line script.
qwertyuio is offline   Reply With Quote
Old 01-19-2020, 03:02 AM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 36,097
Karma: 15014853
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
If i were you i'd setup backup of all my data on a daily schedule including calibre libraries. This should be done once at a system level then you can forget about it.

If you really want to backup when calibre shutsdown then use a script, organize all your libraries under a single folder and backup that folder in your script after calibre quits.
kovidgoyal is offline   Reply With Quote
Advert
Old 01-19-2020, 04:00 AM   #3
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 4,542
Karma: 13300048
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PW4, PW3, Libra H2O, iPad 9.7, iPad 10.5, iPad 12.9
Quote:
Originally Posted by qwertyuio View Post
After messing up a bulk edit I am wondering if it is possible to get calibre to create a backup copy of metadata.db each time it is shutdown. That is, on exit, copy metadata.db to metadata-backup-yymmdd.db (or some other numbering system) up to a maximum number of backups.

This would allow me to restore a previous metadata.db and run check_library to revert to a previous state. I do keep regular backups of my filesystem but these are only run once a day. When I spend an afternoon cleaning up metadata I usually start and stop calibre several times so there would be the potential to save a lot of work reverting to a known good state.

I thought of wrapping calibre in a script to do this but I have several libraries so it is a bit more complicated than a 3 line script.
I actually run automated versioned library backups using FreeFileSync/RealTimeSync *while* Calibre is running (I leave it on 24/7).

I frequently add books remotely (via auto-add folder on Dropbox). RTS monitors the library folders for changes and after 30 minutes of inactivity, runs the backup script: one-way mirror to Dropbox and external drive plus versions on the external drive.
ilovejedd is offline   Reply With Quote
Old 01-19-2020, 07:18 AM   #4
Adoby
Handy Elephant
Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.
 
Adoby's Avatar
 
Posts: 1,610
Karma: 16528970
Join Date: Dec 2009
Location: Southern Sweden, far out in the quiet woods
Device: Thinkpad E595, Ubuntu Mate, Huawei Mediapad 5, Bouye Likebook Plus
It is possible to launch calibre from a script. Make sure the script is found first in the path of folders searched when running commands.

In the script start calibre, using the full path.

After calibre exit, continue the script with the commands needed to backup metadata.db. You may want to add a timestamp to each backup copy and delete the oldest.

I used to do this, but now I just double click a script to backup all my calibre libraries, books and database and all, using rsync snapshots. I keep 7 daily backup snapshots, 4 weekly and 4 monthly snapshots. Since unchanged files are shared among the snapshots it is very fast to update them and they take up very little storage.

I do this between two SSDs in my laptop. Usually takes less than 15 seconds. I also do this, over night, between my laptop and a NAS. Takes a couple of hours. Mostly hardlinking from old snapshots to new snapshot.

So I can travel back in time to retrieve old formats, as needed. Up to 5 months back. (4 weekly
= 1 month.)
Adoby is offline   Reply With Quote
Old 01-19-2020, 07:04 PM   #5
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 15,478
Karma: 16370000
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by qwertyuio View Post
After messing up a bulk edit I am wondering if it is possible to get calibre to create a backup copy of metadata.db each time it is shutdown. That is, on exit, copy metadata.db to metadata-backup-yymmdd.db (or some other numbering system) up to a maximum number of backups.

This would allow me to restore a previous metadata.db and run check_library to revert to a previous state. I do keep regular backups of my filesystem but these are only run once a day. When I spend an afternoon cleaning up metadata I usually start and stop calibre several times so there would be the potential to save a lot of work reverting to a known good state.

I thought of wrapping calibre in a script to do this but I have several libraries so it is a bit more complicated than a 3 line script.
On the basis that Prevention is better than Cure, when I'm unsure of something that I'm about to do in metadata edit search & replace, I copy some relevant books to an empty library using calibre's Copy to library tool, and then test my S&F-fu there. If I stuff up I start over.

I would advise against doing 'cherry pick' backups of calibre libraries or anything similar - e.g your browser's configuration settings. Any time 'saved' doing the backup will be lost many times over when you have to do a restore, the raison d'être of backups. Not only might it be an intricate process, it won't be a familiar process, and you're likely to be in stressed state.

Aside: The check_library tool compares the folder and file names of a library against the database and reports any 'structural' anomalies - e.g. author folders with no matching row in the database authors table, missing/extra formats etc. Some anomalies can be rectified with a button click, whilst others must be done via the file manager. But they all bare 'thinking about' before doing anything.

And if you overwrite library's metadata.db file you may need to (should ??) regenerate the metadata.opf files in the book folders (they are needed by the restore_database tool). This can be done using the calibredb backup_metadata sub command, be aware it can take a while.
BR
BetterRed is offline   Reply With Quote
Advert
Old 01-19-2020, 07:55 PM   #6
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 24,606
Karma: 34133293
Join Date: Aug 2009
Location: The Central Coast of California
Device: K4NT(RIP), Galaxy Tab A, Kobo Aura2
↑ ↑ ↑ ✔ Cherry picking is not advised

metadata.db NEEDS to be in Lockstep with the actual folders and formats.
IMHO use a good file sync app WHEN THE CALIBRE IS NOT RUNNING.
That will sync all Adds, changes, deletes (and skips the no changes)
theducks is online now   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
How to exit calibre? SvenLotus Calibre 4 09-17-2019 10:49 AM
How to exit iomari Calibre 2 08-08-2019 04:37 AM
Aura HD Can't exit settings Mellobob Kobo Reader 15 05-19-2015 08:52 PM
Metadata is changing my Title/author upon exit. Gros'Land Library Management 2 06-04-2011 10:34 AM


All times are GMT -4. The time now is 03:12 PM.


MobileRead.com is a privately owned, operated and funded community.