KF8 and Personal tags

[Kest]

While trying to convert EPUB to AZW3 I noticed that with KindleGen it no longer displays covers on the home screen but it does with Calibre.
Personally I don't really want Calibre in my toolchain so I tried to do it the hard way.

Soon after that, I found that the thumbnails are saved on /mnt/us/system/thumbnails with the format thumbnail_<ASIN>_EBOK_portrait.jpg, and Calibre sideloads the thumbnail there with the appropriate ASIN number, so I tried that with the KindleGen books but without success (Searching the ASIN number with KindleUnpack on the compiled book).

It seems like the problem is that I'm getting the KindleGen books with the "personal" tag, and it somehow ignores the thumbnails (however I don't see that problem with mobi files).

I tried with the full KindleGen output, the kf8-only that I extracted, and also a repacked kf8->epub->kf8 but all of them had the personal tag and ignored the thumbnails.

Supposing that actually the "personal" tag is the problem, is there a way to edit the header or whatever it is on an uncompressed KindleGen output to modify that?

[knc1]

What Kindle firmware version?
There have been some recent changes in that area by Amazon.

[Kest]

JB 5.6.5

[eschwartz]

If a book is a Personal Document, the Kindle will try to extract the cover from the ebook file. This depends on a cover being properly set.

If the book is not a PDOC (that is to say, it has a fake ASIN, or exth 113 field, embedded) then the Kindle attempts to download the cover from Amazon's servers.
Since it depends on Amazon's records and no such book may exist, calibre will generate its own thumbnail from the calibre cover metadata (not the ebook file) and sideload it. There are other tools that will do the same using the embedded cover.

Either way, the end goal of the Kindle is to generate that thumbnail.
The thumbnail for PDOCs is "thumbnail_<random_six_chars>.jpg" so you should probably just confirm that there is an embedded cover.





Apparently fw5.7.2 does away with the highly stupid "Personal" tag. But as far as I know, the above details haven't changed. (I don't own a PW, so I am dependent on hearing things posted here -- but I do try to keep track of other peoples' discoveries.)

[Kest]

Calibre is using the number in its "ASIN" metadata, but definitely is not a valid one (has way more than 10 characters). I don't know about that calibre cover metadata but I guess is a random string associated with both the book and the thumbnail's name (that I tried to emulate without success).

I just noticed that indeed the cover is being extracted from the personal azw3 and saved as "thumbnail_<random_six_chars>.jpg" (really random chars. They re-roll if I cut off then restore the same book from the documents folder), and another one with four random chars if I insert the mobi counterpart of the book.
The problem is that the azw3's cover is not showing (the mobi one showed perfectly from the beginning, even with the personal tag), so maybe really the problem is the personal thing.


Isn't there a tool to modify the properties or whatever it is, of an azw3 to get rid of that Personal Document property?

[knc1]

somewhere here, yes.
mentioned a couple of times very recently.

5.7.2 should also get rid of that irritation.

[eschwartz]

Originally Posted by Kest

Calibre is using the number in its "ASIN" metadata, but definitely is not a valid one (has way more than 10 characters). I don't know about that calibre cover metadata but I guess is a random string associated with both the book and the thumbnail's name (that I tried to emulate without success).

Yes -- that is the problem in the first place. calibre uses UUIDs, since there is really no way to know what the ASIN should be (although if you manually add an amazon:XXXXXXXX identifier, the Quality Check plugin can embed that).

"Since it depends on Amazon's records and no such book may exist"
If the ASIN isn't valid, there is no cover to download.

I just noticed that indeed the cover is being extracted from the personal azw3 and saved as "thumbnail_<random_six_chars>.jpg" (really random chars. They re-roll if I cut off then restore the same book from the documents folder), and another one with four random chars if I insert the mobi counterpart of the book.
The problem is that the azw3's cover is not showing (the mobi one showed perfectly from the beginning, even with the personal tag), so maybe really the problem is the personal thing.


Isn't there a tool to modify the properties or whatever it is, of an azw3 to get rid of that Personal Document property?

Yes, calibre can do it quite easily. The proper settings in MOBI Output, or alternatively the Quality Check plugin I mentioned above.

There is also Java Mobi Metadata Editor, which should be floating around somewhere here.



But I cannot think why the Kindle would be extracting the conver from a PDOC *.azw3 and then not using it.

[Kest]

I tried that Quality Check plugin but only found checking options and some one-click fixers.
The Calibre <format> OUTPUT probably implies a rebuild so I don't think it would be good for testing purposes. I know I could use Calibre but that would kill the initial purpose of not using it in my toolchain

The MOBI editor shouldn't work. I'm not having any problems with mobi files, in fact I'm getting them perfectly with a cover. The ones that didn't work are the azw3/KF8 ones.


Indeed I'm also clueless about why the thumbnails are extracted but not used. Maybe I'm doing something wrong, so anyone posting their experience with thumbnails and KindleGen generated azw3/kf8 books on Kindle would be appreciated.

[knc1]

Originally Posted by Kest

I tried that Quality Check plugin but only found checking options and some one-click fixers.
The Calibre <format> OUTPUT probably implies a rebuild so I don't think it would be good for testing purposes. I know I could use Calibre but that would kill the initial purpose of not using it in my toolchain

The MOBI editor shouldn't work. I'm not having any problems with mobi files, in fact I'm getting them perfectly with a cover. The ones that didn't work are the azw3/KF8 ones.


Indeed I'm also clueless about why the thumbnails are extracted but not used. Maybe I'm doing something wrong, so anyone posting their experience with thumbnails and KindleGen generated azw3/kf8 books on Kindle would be appreciated.

I am pretty sure this site has a forum for those types of topics.
Browse the 'home' page index a bit.

[NiLuJe]

@Kest: To make a long story short, you'll only ever get proper thumbnails by making sure there's an ASIN set in the mobi headers, and sideloading said thumbnail yourself (i.e., what Calibre does).

Any other combination has one drawback or another (and some even make the Kindle framework do stupid things, cf. the discussion in the last few messages on the ScreenSavers Hack thread).

If you don't want to use Calibre, I'm fairly sure there are other tools than can do that, but I'm not as familiar with them.

----

As for the cover being extracted but not being used, that can happen a number of ways (cdeKey clashes, missing cdeKeys, and/or weird caching behavior of the framework, depending on how your test files where transferred/deleted).

[eschwartz]

Originally Posted by Kest

I tried that Quality Check plugin but only found checking options and some one-click fixers.

... for example, Fix ==> Fix ASIN for Kindle Fires -- right?

The Calibre <format> OUTPUT probably implies a rebuild so I don't think it would be good for testing purposes. I know I could use Calibre but that would kill the initial purpose of not using it in my toolchain

Toolchain or not, all it takes is resending the book, since that particular option involves metadata which calibre updates during send-to-device.
Note: Conversion settings are cached per book record, Bulk Edit Metadata has an option to clear it.

The MOBI editor shouldn't work. I'm not having any problems with mobi files, in fact I'm getting them perfectly with a cover. The ones that didn't work are the azw3/KF8 ones.

JMME works on both MOBI and AZW3 -- they use the same binary wrapper.


...

You could also investigate the code calibre/Quality Check uses and incorporate that into a standalone utility for the sake of your toolchain.

Regardless -- any of these solutions will allow you to embed an ASIN in the exth 113 field, at which point you will have very predictable thumbnail files that you yourself will have to add.