opf files and managing metadata

[please55]

So after a previous google search it would appear I've misunderstood metadata management all along... This isn't quite like managing my mp3 collection.

Are opf files just metadata containers? What about the calibre .db file, isn't that a metadata container? Where are all the metadata updates I'm doing in calibre being saved to?

If I have a folder that contains an epup, an opf file, and jpg what is the correct method to instruct Calibre to import the metadata / cover image as well when adding the book?

[itimpi]

Originally Posted by please55

So after a previous google search it would appear I've misunderstood metadata management all along... This isn't quite like managing my mp3 collection.

Are opf files just metadata containers? What about the calibre .db file, isn't that a metadata container? Where are all the metadata updates I'm doing in calibre being saved to?

If I have a folder that contains an epup, an opf file, and jpg what is the correct method to instruct Calibre to import the metadata / cover image as well when adding the book?

Calibre writes metadata to its own database (the metadata.db file). It then creates the .opf file with a copy of the metadata stored in the calibre database and stores it with the ebook files. The .opf file is intended to be used for recovery purposes if for any reason the main database (metadata.db) gets corrupted.

What metadata can be stored inside an ebook files varies according to the capabilities of the format in use. epub books for instance have an equivalent to the .opf file embedded in them.

[theducks]

:adding to the above:

The book stored, does not automagically receive the new metadata when you make a change in Calibre.
The stored book only changes (if supported by format), if you use one of the tools that causes the book to be modified

Books 'Sent' (exported) get updated on the fly

[please55]

Thanks everyone.

Originally Posted by itimpi

The .opf file is intended to be used for recovery purposes if for any reason the main database (metadata.db) gets corrupted.

In that scenario, how would you recover the metadata from the opf file and sync it with each book?

Originally Posted by theducks

The stored book only changes (if supported by format), if you use one of the tools that causes the book to be modified

Are there any other tools that save the info right to the book as is? Is all metadata saved (including custom fields you might have created in calibre)?

Originally Posted by please55

If I have a folder that contains an epup, an opf file, and jpg what is the correct method to instruct Calibre to import the metadata / cover image as well when adding the book?

Any insight on this particular? Does the add book from directories do this or does that just look for ebooks and ignore opf files in the folder?

[BetterRed]

Originally Posted by please55

In that scenario, how would you recover the metadata from the opf file and sync it with each book?

Are there any other tools that save the info right to the book as is? Is all metadata saved (including custom fields you might have created in calibre)?

Any insight on this particular? Does the add book from directories do this or does that just look for ebooks and ignore opf files in the folder?

@please55 - forget about the metadata.opf files, they are for calibres internal use if/when a database (metadata.opf) needs to be rebuilt. Furthermore at any given point in time they might NOT contain the very latest metadata that's in the database, this because the writing of the opfs runs in background at low priority. Pretend they don't exist.

You can populate the database from the metadata within a book thus:

1. when you add it to the library, in Preferences->Add Books you define whether metadata is to be extracted from the book file or from the file name.
   
2. in Metadata edit, by selecting a format and hitting the blue I icon

You can populate the the metadata data in a book in several ways

1. by Sending a format (epub, mobi, pdf etc) to a device, or Saving it to disk, in which case the file sent/saved will contain such metadata that the format is capable of having within it - eg epubs can have more than MOBI and PDF's.
   
   But the format files in the library itself are not changed - many users prefer to leave the format files in the library untouched.
   
2. by using the Embed Metadata(most formats), Polish Book (EPUB and AZW only) or Modify (EPUB only) plugins. These will update the format files in the library itself.
   
   Conversion will also update the metadata in the output file - but if you just want to update the metadata in an existing format then I would suggest using one of the other three tools, conversions are slower and can potentially alter the content of the book.

I can't answer the issue of adding books from a folder that contains an opf. Mainly because I can't recall any books I've purchased ever having them, I thought they were peculiar to calibre - especially if they're named metadata.opf.

BR

[please55]

@BetterRed - thanks. My concern was if somehow I was to lose all the work I've done in calibre library there doesn't appear to be a way to recover.

Unlike mp3 management where for the most part changes you make are saved to the mp3 itself and there's no chance of losing your work if the software (say mp3tag) stop working, corrupted, wasn't available.

[itimpi]

Originally Posted by please55

@BetterRed - thanks. My concern was if somehow I was to lose all the work I've done in calibre library there doesn't appear to be a way to recover.

That is the purpose of the .opf files. Calibre has a facility to rebuild its database by reading the file system and finding all the .opf files it has stored with the ebook files.

[theducks]

Library Maintenance:Rebuild reads those files and creates a new metadata.db.

AS LONG AS YOU NO MESS WITH THE LIBRARY

OTOH if you do something dumb (using Calibre), like deleting all Tags.
Those OPF files start updating immediately. (see Library Maintenance:Metadata backup status: <it shows remaining to be written> count> )
Do NOT use the button on a damaged DB, That forces the BAD DB's values to be written to OPF's

[eschwartz]

In addition to the Restore Database tool (run from within a broken library, see Library Maintenance in the dropdown by the Library icon in the toolbar) you can also use the second option in Add books -- it imports one book per directory, assumes each file is the same book in a different format. What this means is if you Save to disk or even point Add books at a different library, it will read metadata from the metadata.opf and cover.jpg.

[please55]

Thanks everyone.

**Edit**
One final question about metadata. If you create custom fields I imagine those aren't saved to the file even when a conversion / polish is done. Is there somewhere to look up the set of metadata that is stored for epub files?

[davidfor]

Actually, the custom columns are written to the metadata in the books. Another calibre installation will read them when you put the books in the library, but not many other applications will.

[BetterRed]

Originally Posted by please55

Thanks everyone.

**Edit**
One final question about metadata. If you create custom fields I imagine those aren't saved to the file even when a conversion / polish is done. Is there somewhere to look up the set of metadata that is stored for epub files?

Yes, custom columns are saved too, you can examine the metadata.opf files using a text editor, but don't change them in any way.

If you want to know about OPF start here ==>> International Digital Publishing Forum | The Trade and Standards Organization for the Digital Publishing Industry

BR