Trouble editing Project Gutenberg books

[gol8erl8]

I appreciate you testing for me, theducks.

Originally Posted by theducks

Download EPUB from PG, (optional edit) ,then convert to AZW3 (KF8) with Calibre. Mobi is Old and limited. All Kindle tablets do the New format

Originally Posted by hobnail

Given the choice of formats to download from any site my theory has been that downloading EPUB is always a safe bet and Calibre will do an excellent job of converting it to Kindle format (most likely you'll want AZW3, not MOBI unless your Kindle is ancient). Additionally during the conversion you can have Calibre fix any wonky formatting that may be present in the EPUB, which in the case of Project Gutenberg (and probably the other sites as well), will also be present in their Kindle version.

I will take your and hobnail's suggestion of starting with EPUB files. I only went with MOBI because I wasn't planning on converting.

Originally Posted by hobnail

For the OP, I think Kovid meant EPUBs where he typed EPBs.

Thanks.

[hobnail]

Originally Posted by gol8erl8

Thank you and any other developers for calibre. It is great software. I'm curious what a "dual" MOBI format is. Since other MOBI files did work, I had reason to believe there might be multiple types.

Dual mobi contains 2 different formats, the old original mobi aka mobi7 I think it's called, as well as the newer azw3. It therefore works on both very old kindles as well as the newer ones, each only seeing the format they need to. You can also create a dual mobi with the program kindlegen, feeding it an epub, and the output of that also contains the original epub as well as the old mobi and azw3 so it's more of a triple mobi. I don't know which type of mobi calibre creates with its different output options for mobi. If you're making books to sell on amazon then the mobi that kindlegen creates is what's wanted. For yourself you don't need it unless you have an old kindle; in that case calibre can probably produce a single mobi7 format mobi.

[gol8erl8]

Originally Posted by BetterRed

@hobnail - yes, when you Send to Device or Save to Disk you can send/save with the current metadata - IIRC StD metadata setting is device dependent. You can also inject the current metadata into the library copies of EPUBs (and AZW3s) with the Polish Books tool. There's also an Embed Metadata tool.
Attachment 177311

BR

Originally Posted by retiredbiker

For years my workflow has been add book, fix metadata, Polish. For some small change if I spot a rock, like changing a tag, immediately hit Embed Metadata. Doing this, I've never had a problem with what is actually in the book compared to what I see on the Calibre screen.

(I keep Polish set to update the metadata, and also update the cover, smarten punctuation, and remove unused css rules. Keeps everything nice and clean.)

Yesterday, I found extra tools, including Polish books and Embed metadata, under Preferences > Toolbars & menus. I am happy to see their use advocated for here. However, I have some trouble that I'll explain by example:

When I add https://www.gutenberg.org/ebooks/61436.kindle.noimages to my library, clear the metadata using Edit metadata in bulk, convert to .AZW3, Save to disk, and then add the new file to my library, the cover and the Date and Languages fields are repopulated. Moreover, if I convert before clearing the metadata, the Tags field is repopulated, but strangely only with the last tag from the original file. This also happens when I use Polish book or Embed metadata after clearing the metadata but before saving to disk. Is it possible to make these (and all fields, in general) blank if they were originally populated?

[gol8erl8]

Originally Posted by hobnail

Dual mobi contains 2 different formats, the old original mobi aka mobi7 I think it's called, as well as the newer azw3. It therefore works on both very old kindles as well as the newer ones, each only seeing the format they need to. You can also create a dual mobi with the program kindlegen, feeding it an epub, and the output of that also contains the original epub as well as the old mobi and azw3 so it's more of a triple mobi. I don't know which type of mobi calibre creates with its different output options for mobi. If you're making books to sell on amazon then the mobi that kindlegen creates is what's wanted. For yourself you don't need it unless you have an old kindle; in that case calibre can probably produce a single mobi7 format mobi.

Neat. More great information, hobnail.

[gol8erl8]

Correction: https://www.gutenberg.org/ebooks/76.kindle.images actually has tags to test with.

[BetterRed]

Originally Posted by gol8erl8

Yesterday, I found extra tools, including Polish books and Embed metadata, under Preferences > Toolbars & menus. I am happy to see their use advocated for here. However, I have some trouble that I'll explain by example:

When I add https://www.gutenberg.org/ebooks/61436.kindle.noimages to my library, clear the metadata using Edit metadata in bulk, convert to .AZW3, Save to disk, and then add the new file to my library, the cover and the Date and Languages fields are repopulated. Moreover, if I convert before clearing the metadata, the Tags field is repopulated, but strangely only with the last tag from the original file. This also happens when I use Polish book or Embed metadata after clearing the metadata but before saving to disk. Is it possible to make these (and all fields, in general) blank if they were originally populated?

Without know the settings you have in Add Books, Save to Disk, Convert etc, as well as what plugins you have installed and how they're configured, its hard to offer any definitive comment.

I clicked the pg link you provided, which was a download of pg61436.mobi.

When I added that file (pg61436.mobi) to my test library the only metadata columns that were not empty were title, author, date, published date, and language I converted it to AZW3, saved that to disk, as far as I can tell its metadata is similarly sparse.

I'm a bit puzzled by the step I've changed to red.

Why are you saving something and then adding it back to the library? If you convert a format (e.g. MOBI) that's in a library book folder the output (e.g. AZW3) is written to the book folder -- so why save it and add it back.

And, given I don't get any 'optional' metadata like tags, comments etc for that specific book I don't understand why you are finding it necessary to even go near Bulk Metadata Edit.

Added: we Cross Posted. I'll have a look at the new link.

BR

[BetterRed]

OK I just did the same thing with the Huck Finn, except immediately after downloading I cleared the tags as I would normally -- by pressing F2 in the Tags cell in the book list, pressing the Del key, then pressing Enter twice.

I then converted the MOBI to AZW3, and again as far as I can tell it has no tags.

As I would expect the MOBI still has them. If I wanted to get rid of the tags in the MOBI I'd try Embed Metadata, if that didn't work I'd try MOBI->MOBI convert, if that didn't work I'd probably give up and throw the MOBI away.

Why are you downloading MOBIs. If PG has the books I want in EPUB that's what I download.

BR

[gol8erl8]

@BetterRed I appreciate you taking the time to help me! Allow me to answer your questions in order:

---
I am using a virtual machine snapshot, so my calibre settings are all defaults, except for Preferences > Saving books to disk > uncheck Save cover separately, uncheck Save metadata in a separate OPF file, Save template: {title} - {authors}. When saving to disk, I select Save single format to disk... > AZW3. Nothing is changed when converting books, and I have no plugins. I'd be happy to screenshot something, such as Add books settings (which are defaults), if requested.

I would not normally add the exported file back to the library, making an unnecessary duplicate (or triplicate if counting the exported file and now two files in the library). I am only doing this to exam the metadata of files calibre exports (via Save to disk and Send to device). If someone said that, I would be puzzled too.

I am using Edit metadata in bulk because, in reality, I have nearly 200 files for which I want to clear all the metadata for, except Title and Author, because it is incorrect or badly typed. I am just trying to state my process accurately in my example (save the number of files). Thanks for the F2 tip!

As aforementioned, I did not plan on converting, and my Kindle does not read .EPUB. I will be redownloading all of my files as .EPUB soon. However, I found that I have the same problem when I tested on the .EPUB version of Huck Finn.
---

To be clear, did you check the metadata of the exported .AZW3 file by adding it into the test library? For me, the metadata for the original book record is cleared, of course. It is the exported file that gets repopulated with the junk (Tags: Mississippi River -- Fiction, Date: 27 Feb 2020, Languages: English). Also, it matters whether I save with the .OPF file. Moving/deleting the .OPF from the same folder as the exported .AZW3 file (or using Save to disk with Save metadata in a separate OPF file unchecked) before adding it back to the library might be the difference in our results.

[BetterRed]

I still see no point in adding something that's already there. If you no longer need the format you got from PG (e.g MOBI) use the Remove specific format option to get rid of it.

But here's the rub: you are doing bulk metadata edits, but you probably haven't realised that the metadata.opf files are written to the book folders by a low priority task that executes in the background, on my system it chugs along at about one book per second.

And I suspect you are doing a Save to Disk, that includes the .opf, before the fresh metadata.opf file has been written to the book file.

And I suspect you are then adding the books back with one of the Add options that uses the metadata.opf files as buddies to the azw3 files.

See first paragraph

BR