Custom Columns - are they written to epub?

[theducks]

Originally Posted by judyni

I installed sigil and it doesn't "see" the custom columns in its metadata editor. Of course I can edit the metadata.opf by hand in either calibre or sigil but I don't want to do that. So my custom columns are there but not easily visible...

IIRC Sigils metadata editor only shows Known DC tags
You need to Look directly in the OPF file, They are ther, but not as the simple encoding as the DC type get

[judyni]

Thanks, Chaley, but that's more work than I'm willing to put into it. I installed Modify ePub but it also doesn't have the option not to include (or to delete) Custom Columns. I think it will only update epub files. Polish doesn't seem to have the option not to modify the AZW files at the same time.

[BetterRed]

Originally Posted by judyni

Darn! I get it now. The entire metadata.opf is in there. The custom columns are not visible in the metadata display, but they are there nonetheless.

I do want all the standard metadata but not the "custom columns". I don't think that's an option. I think it should be. Surely I'm not the only one who doesn't want all of my comments to be attached to a book someone else might read.

So my plan isn't going to accomplish what I wanted.

@judyni - I should clarify what DoctorOhh & I discovered

If you create an EPUB via conversion then the embedded metadata is the minimal set. But if you do any of the following (there may be others) then the metadata in the book's EPUB file will be updated with all the book's metadata in the database (I'm not sure about other formats)

• Polish with Update Metadata set
• Modify with Update Metadata set
• Edit Book with Preferences->Integration with calibre->Update metadata embedded in book when opening checked
• Send to Device or Attached Folder
• Copy to Disk

The reason I don't see any custom metadata in my EPUB's is because I do not do any of the above. My reasons for not doing them are various, none of which is to avoid getting custom columns embedded as metadata - that's serendipitous.

To prove it to yourself I suggest doing an EPUB to EPUB conversion and looking at the EPUB's content.opf via Unpack or Sigil.

BR

[judyni]

Originally Posted by BetterRed

@judyni - I should clarify what DoctorOhh & I discovered

If you create an EPUB via conversion then the embedded metadata is the minimal set. But if you do any of the following (there may be others) then the metadata in the book's EPUB file will be updated with all the book's metadata in the database (I'm not sure about other formats)

BR

Very curious. You say look at "unpack" (don't know what that is) or sigil, but wouldn't Calibre's own Edit Epub show the same thing? Surely it wouldn't "cheat" and look at the metadata.opf file...

I just spent the day rewriting my Epubs with Polish (or Modify Epu if there was an AZW). Sigh...

[BetterRed]

@judyni Unpack Book is the old Tweak Book rebadged - select a book that has an EPUB and press 'U', click Explode Book and you have the content.opf to look at (I use Notepad++ because it comprehends XML)

Re calibre editor - no it doesn't cheat, but if you have that box checked to update the metadata when a file is opened then that's what it will happen - IIRC its on by default.

BR

[DoctorOhh]

Originally Posted by judyni

Very curious. You say look at "unpack" (don't know what that is) or sigil, but wouldn't Calibre's own Edit Epub show the same thing? Surely it wouldn't "cheat" and look at the metadata.opf file...

But it would and does essentially look at the metadata and update the book upon opening it in the editor. This happens if you have the Update metadata embedded in the book when opening item checked in Edit Book's Preferences (which is enabled by default).

[BetterRed]

@judyni

I know of no convenient way to remove 'non standard' metadata in a book, other than by editing the content.opf by hand or by doing an EPUB to EPUB conversion. But if you do a Save to Disk or Send to Device of that EPUB you'll get it all back again.

What Rot - See post #24 - BR

So assuming you had a EPUB unpolluted by superfluous metadata ('pristine') then what do you want to do with it - if its send/upload to an external entity, then my advice is to copy the 'pristine' EPUB out of the book folder with your file manager to somewhere like your desktop and then deal with it using dumb tools like your operating system, browser, mail client etc.

BR

[judyni]

This is not so important as to go to extreme means to keep the custom metadata out. I just would prefer my comments not be in the epub files. Some might be useful, like if I have an associated audible file, but since we can't see them anyway without an epub editor, it doesn't help for it to be there.

I experimented before the above last few replies and I thought I was going nuts! Who would have thought something like "Update metadata embedded in the book when opening" would be even there much less the default. Once that is in there, I can't see a way to get the metadata out again short of editing the opf file. Even what you think of as the original epub is not if you opened it in the editor because of that setting. Furthermore, you're going to pollute at least one file to change the setting, since the setting is in the editor itself. I think I remember glancing at the settings and didn't see anything dangerous, but being a novice with Calibre it's hard to know what's going on. I probably didn't even get to the last line -- integration with calibre.

SO... Yes, after I figured out I was not going crazy and didn't touch the calibre editor any more and used sigil, the custom metadata is not written out with Convert Books. I'm not sure this is trustworthy though because it could be an "oversight" that might be "fixed" at any time, since the other methods of writing out metadata write it all out...

Thanks everyone!
Judy

[DoctorOhh]

Originally Posted by BetterRed

I know of no convenient way to remove 'non standard' metadata in a book, other than by editing the content.opf by hand or by doing an EPUB to EPUB conversion.

I believe the Modify Epub plugin allows you to remove Non DC metadata which does remove the custom column info.

[BetterRed]

Originally Posted by DoctorOhh

I believe the Modify Epub plugin allows you to remove Non DC metadata which does remove the custom column info.

DoctorOhh - you're right again, why didn't I notice that I even have it checked ! Familiarity, contempt... For me Modify is menu key, up arrow, enter, enter.

@judyni - I normally don't have anything in Comments (dc:description) most of my books are technical 'papers' from academia, government, think tanks, media etc. I put my comments in a long text custom column called Notes, I do that because I can edit it from the book list cells rather than having to use the metadata editor. Hence my Notes don't go into content.opf.

So, if you use Modify with Update metadata unchecked, and Remove non dc: metadata elements checked then you'll have what I referred to as 'pristine' EPUB.

If you don't the want the contents of Comments to go into dc:description then don't use Comments, instead use a custom column eg Notes.

I think you should be able to use Bulk Metadata Edit Search and Replace to move what you have in Comments to a Notes column. But that's not something in which I have sufficient knowledge to offer detailed advice.

Re the setting in the editor - yes I was surprised by it too, and that the default was True. I think I discovered it within a day or so of the editor being officially released. My backup reported an EPUB as being changed and I was darned sure hadn't changed it, because I would have remembered saving it. I reported the error, Kovid bought the setting to my attention and I immediately turned it off. It was in my Test library so it was of little consequence.

Originally Posted by judyni

I'm not sure this is trustworthy though because it could be an "oversight" that might be "fixed" at any time, since the other methods of writing out metadata write it all out...

I'd be surprised if this was not a concious decision on Kovid's part, hence I would be surprised if it changed. But it doesn't matter because we now know it can be fixed with the Modify Plug In.

BR

[judyni]

Modify ePub on multiple files followed by Remove Books > Remove files of a specific format made short order of getting the custom columns out of my epubs...

Thanks everyone! I learned so much about calibre from this thread.

[BetterRed]

Originally Posted by judyni

Modify ePub on multiple files followed by Remove Books > Remove files of a specific format made short order of getting the custom columns out of my epubs...

Thanks everyone! I learned so much about calibre from this thread.

@judyni - I feel I owe you a bit of an apology, for all the confusion and dead ends. Your request to have 'clean' EPUBs was an unusual request, the first I can recall. I knew it was possible because I had 30,000+ of them but I wasn't really sure what I was doing or not doing to get them.

But we got there in the end and I too learnt some things. Thanks for persevering with it, and for letting us know that you now have what you wanted.

BR