View Single Post
Old 05-26-2010, 12:30 AM   #19
q345
Member
q345 began at the beginning.
 
Posts: 23
Karma: 10
Join Date: May 2010
Device: none
"spread over" duplication of metadata in the e-book directory proposal

Chaley, I wasn't aware ( being a new to Calibre - installed it a few days ago by accidentally "bumping" into it on the web) about future plans (plug-in storage), which might affect the design in a very substantial way. ( I am guessing one will be able to put the whole Calibre Directory on USB stick ).

There is a fundamental difference between Calibre and digital music apps, namely that metadata and data in mp3/flac/ogg are kept in the same file, so metadata in such a case is completely duplicated, hence trivially reconstructable by simply rescanning the directory tree with original files.

As you pointed out it's not the case with e-book/Calaibre where metadata is NOT part of the original file, so in case of metadata.db get corrupted/get out-of-sync, the restoration might be cumbersome at best and impossible at worst. For a person, who use Calibre to keep tracks on thousand of books/papers and has invested considerable amount of time over months and even years into creating metadata. that might be indeed a huge loss.

One of the possible solutions might be (obviously) to duplicate metadata by keeping all tags/info of a given instance of e-book in a separate small file in the Calibre directory where the e-book (all formats of it) itself is kept, IN ADDITION to metadata.db file itself. In case metadata.db file got corrupted, restoration will be a simple process of scanning the Calibre directory tree, reading all those small metadata files each of them is related to a single e-book only ( to be precise to all the formats of a single e-book) and rebuilding a "main" metadata.db file.

It would be interested to hear Kovid opinion if this idea has merit . In essence it's a back-up of metadata.db file "spreaded over" the whole e-book tree. It might also made the whole product more robust, regardless of NAS,
since even if few "small" metadata files are lost, it still will be possible to recreate the bulk of metadata.db automatically. This scheme is conceptually identical to digital music numerous apps , all of them simply put original music tags into database for performance/searchability reason, but it's also possible navigate a music collection by browsing original directory tree.

Last edited by q345; 05-26-2010 at 12:33 AM.
q345 is offline   Reply With Quote