Transferring metadata from one file to another

[MAMEBase]

Hello -

Apologies if this has been answered elsewhere, or if I'm simply missing something obvious.

I've recently come into possession of a fairly large library of badly formatted ebooks (ePub files) that I am in the process of correcting (properly formatting chapters, that sort of thing).

My problem is that in generating the new, corrected ePub files, the metadata from the old files does not accompany the new files (not surprising, or expected).

My question is this: Is there an easy way (within Calibre) to transfer the metadata from one file to another directly?

[davidfor]

I think you want to write the metadata from the calibre library to the actual ebook. There are two ways I can think of to do this.

As you are using ePubs, get the Modify ePub plugin. This has an option to do this. This they way I usually do this.

The other way is via a conversion. calibre will update the metadata when you do a conversion. As you are doing a cleanup of the books, starting with a epub-to-epub conversion could be a good idea.

[MAMEBase]

Thank you, I appreciate your reply.

I've obtained the Modify ePub Plugin, but I'm not certain it's going to do what I need it to do.

Let me explain my process here -

The ePub Files in question are the result of scanned books, and as such contain various formatting difficulties, including, but not limited to odd line breaks, improper/missing chapter markers, misplaced letters due to drop caps, and embedded page numbers.

To correct these problems, I'm extracting the text from the documents (converting to text), and extracting any illustrations (including Cover art).

I then make any needed corrections to the text file with a word processor (Actually, I simply use TextEdit), and reprocess the file into an ePub via Legend Maker.

The one good thing about the original ePub files, is the embedded metadata that appears in Calibre. It's very detailed, and I'd like to retain it in my newly generated ePub files.

My first pass at the collection will likely not be the last. My hope is to eventually share the collection with others who may see fit to refine the corrections further, and to that end, I plan on making both the ePub file(s) as well as the source text files available. As the source text is refined, and new ePubs are generated, I want to be able to transfer the metadata from the existing ePub file(s) to the new one(s).

I've done something similar in the past with MP4 files and iTunes.

I once made up a set of 'Placeholder' files (very small MP4 files that contain a few seconds of instructional text as video) to hold very detailed metadata about the original 'Star Trek' series.

When/If I obtain a better quality video file for an episode, I can then transfer the metadata to the file using Subler, or MetaX.

I'd like to do something similar with Calibre.

I can easily generate a nearly empty ePub file (perhaps containing instructions for use), and use that file to hold the metadata in its proper format, but I need a mechanism to transfer said metadata from one ePub file to another.

Is there a plugin for that?

[davidfor]

OK, I see what you are doing.

The simplest solution for this is after you create your new ePub externally, select the old one in calibre, and drop the new one into the details pane of the display. calibre will then replace the existing ePub with this new one. Then you can use the Modify ePub plugin to update the metadata in the new ePub. You can also do this while editing the medatada. Drop the new file in the list of formats or use the add button to find it.

A few things I can think to add to what you are doing:

- You can add books to calibre without a format. Or delete the format after adding it in. This is the equivalent of your placeholder in iTunes but calibre handles it properly.

- Have you considered using Sigil to edit the original ePubs rather than extracting the text?

- I would probably use multiple calibre libraries for this process. One would hold the originals with any metadata cleanup. I would copy this entry to another calibre library (using calibre) and then clean the ePub. When satisfied with the cleaning, I would move them to a third permanent library. I occasionally do something like this, but I skip the first library and keep the original external to calibre until I am happy.

[paw3pals]

Would it be possible to add the newly formatted book to Calibre, say Yes to adding a duplicate, then just merge them (right click, select Edit metadata, select Merge book records)? Or would them being the same format be a problem?

Originally Posted by davidfor

OK, I see what you are doing.

The simplest solution for this is after you create your new ePub externally, select the old one in calibre, and drop the new one into the details pane of the display. calibre will then replace the existing ePub with this new one. Then you can use the Modify ePub plugin to update the metadata in the new ePub. You can also do this while editing the medatada. Drop the new file in the list of formats or use the add button to find it.

A few things I can think to add to what you are doing:

- You can add books to calibre without a format. Or delete the format after adding it in. This is the equivalent of your placeholder in iTunes but calibre handles it properly.

- Have you considered using Sigil to edit the original ePubs rather than extracting the text?

- I would probably use multiple calibre libraries for this process. One would hold the originals with any metadata cleanup. I would copy this entry to another calibre library (using calibre) and then clean the ePub. When satisfied with the cleaning, I would move them to a third permanent library. I occasionally do something like this, but I skip the first library and keep the original external to calibre until I am happy.

[MAMEBase]

I believe I am beginning to understand how the metadata is handled in Calibre.

It looks like the metadata tags are stored in three separate places. The main Calibre database, the EPUB file itself, and the external OPF file.(Sometimes it pays to read the manual ).

Now that I understand this I've found a solution (albeit a clunky one) -

Step one – Make a new Calibre library, and add newly generated EPUB file(s) to the library.

Step two – Open the path to the new EPUB file(s) and remove the metadata.opf file(s).

Step three – Switch to the original library, and locate the original EPUB file(s).

Step four – Open the path to the original EPUB file, and copy the metadata.opf file(s) to the corresponding folder(s) in the new library.

Step five – Switch back to the new library, and restore the database.

Step six – Repeat steps two through five as needed.

The modify EPUB plug-in appears to be able to write to the metadata.opf file, but not read from it (still very useful in other ways, though).

FWIW, I would love to see a plug-in that reads the metadata embedded in the EPUB file and the metadata.opf file and allows one to compare the two, make changes in either location, synchronize the data (in either direction), and update the Calibre database.

I've considered using Sigil to edit the existing EPUB files, but in this instance, I decided it would be easier to create a new structure, rather than attempt to patch the old one.

I'd like to thank you again for your response, and your assistance. You pointed me in the right direction, and as a result, I've learned a little bit more about the EPUB format.

[kiwidude]

Maybe I am missing something here, but that just all sounds rather horrible?

You have a library which has the correct metadata, correct?

Why not either (a) Just drop your "new" ePub into that library replacing the old one (either drag/drop into book details panel or edit metadata screen formats panel) and then run Modify ePub to get the internal ePub metadata up to date (an optional step).

Or (b) if you want to keep a copy of your original epubs, just copy the whole library to a new library, remove all the book formats (using the Remove menu) and then just do the same step in (a) of drag/dropping in the new ePub,

or (c) create the second library as blank, then as you create each new epub version you use "Copy to Library" on the right-click menu to copy the book/metadata across and then do (a) above. This option means your "new" library only has books you have converted.

Fiddling with calibre's opf files and restoring library etc all sounds very nasty and you really shouldn't go spelunking in calibre's library folders like that.

Using Sigil to just edit the ePubs "in place" is by far the easiest approach of all (particularly when done with the Open With plugin) and is certainly the approach that most senior calibre users like myself do. Going to text files and thereby losing all your formatting and various other stuff before going back to ePub sounds a bit mad, unless the books are the most woeful of pdf conversions. But hey, whatver method you are happy with

[davidfor]

Originally Posted by paw3pals

Would it be possible to add the newly formatted book to Calibre, say Yes to adding a duplicate, then just merge them (right click, select Edit metadata, select Merge book records)? Or would them being the same format be a problem?

I think this would work, but I haven't used the Merge function enough to know exactly how it works. But, I can't think of an advantage in doing it this way other than not needing to find the book in calibre before replacing it. I suppose it would be handy if you have a lot of books to replace. After adding the duplicates you could use the Find Duplicates plugin help with the merges.

[davidfor]

Originally Posted by MAMEBase

Now that I understand this I've found a solution (albeit a clunky one)

I have to agree, "clunky". And I think prone to failure.

I think kiwidude has it right. I was sort of saying the same thing, but he has explained it in better detail.

[MAMEBase]

Okay, clearly I'm an idiot.

Once I realized what you meant by the "details panel", I got things to work the way you described.

Thank you again for your insight, this could've all gone very badly.

[travger]

Originally Posted by paw3pals

Would it be possible to add the newly formatted book to Calibre, say Yes to adding a duplicate, then just merge them (right click, select Edit metadata, select Merge book records)? Or would them being the same format be a problem?

Yes, it will be a problem. Calibre allows only one file of a same format under the book entry, so if you want to keep three epubs of Book X, you need three entrys. If you just want to replace the epub or add mobi to existing epub, it's enough to drag-drop new file into book details - in first case it overwrites existing epub, in latter case adds mobi format.
I have my prefs set so that existing format will be added as duplicate entry, only eyeballing can say which one to delete.

For merging, hit just 'm' on the keyboard - and read the text in the pop-up window.