I am glad you managed to rebuild your library. That is certainly the important thing.
Thank you for posting. It is always useful to have clear and concise feedback on a process.
Quote:
Originally Posted by MiniMouse
- When you try 'Library Maintenance → Restore database' you have to rebuilt ALL you custom columns and templates first, otherwise you will only get the default installed columns filled (because calibre can't write any data in columns that don't exist even when the data for them is stored in the books metadata).  A point that wasn't made really clear for me in the calibre manual. |
When restore is run, it checks for the existence of the file metadata_db_prefs_backup.json. If that file is there, it is assumed to contain the correct custom column definitions. If it is not there or if it is corrupted, restore will scan every book looking for metadata.opf files and rebuild the custom column information from what it finds.
To get the behavior you saw, three things must be true:
1) there was no metadata.db file in your library folder, and
2) calibre successfully initialized a database in that folder, and
3) calibre successfully closed, which is when metadata_db_prefs_backup.json is written.
This sequence of steps will overwrite the metadata_db_prefs_backup.json file with initial values, preventing the restore of all the custom column information. You can force calibre to do the scan by deleting the metadata_db_prefs_backup.json file then running the restore without closing calibre (from the command line would be good). Of course, you wouldn't know to do this.
One does wonder why your library metadata.db disappeared ...
One reason that metadata_db_prefs_backup.json exists is that custom column changes take time to appear in all the metadata.opf files. Until all the files are brought up to date, the scan will produce conflicting information. Calibre attempts to handle those conflicts by assuming that the newest metadata.opf contains the correct information, but that might not be true if no metadata.opf files were written or if the file is corrupted.
I wonder if I shouldn't change calibre to do both the scan and the load from metadata_db_prefs_backup.json. It isn't clear what to do with the results. My feeling is:
a) a column definition (a lookup key) found in metadata_db_prefs_backup.json should win over the same definition found in a metadata.opf, and
b) a column definition found in the scan that is not in metadata_db_prefs_backup.json should be added to the db.
This scheme could re-add old custom columns to the db, but that is easily fixed once the recovery finished. It would also deal with the problem you must have had, where the metadata_db_prefs_backup.json file was reinitialized somehow before the recovery was run.