Kindlegen mobi display issues

[marnold]

I'm trying to convert a book (see sample-ebook.pub) to Kindle (mobi) using KindleGen that is shipped with the latest Kindle Previewer 3.43.0. I transfer the resulting mobi via Send to Kindle app (version 1.1.1.250) on a Mac. The book is going to be distributed without KDP.

I found a couple of puzzling display issues on different devices:

1. Cover is displayed on Kindle for iOS (see cover_ios.jpg) but not on Kindle Voyage (see cover_kindle.jpg)

2. Page formatting is fine on Kindle Voyage (page_kindle.jpg) and Kindle Previewer (page_previewer.png), and broken on Kindle iOS (page_ios.jpg)

Any clues on how to debug/fix this?

[jhowell]

Originally Posted by marnold

I'm trying to convert a book (see sample-ebook.pub) to Kindle (mobi) using KindleGen that is shipped with the latest Kindle Previewer 3.43.0. I transfer the resulting mobi via Send to Kindle app (version 1.1.1.250) on a Mac.

A MOBI file created by KindleGen is a Master MOBI that contains both the older MOBI7 and the newer KF8 formats in a single file. Different Kindle apps/devices will use either one or the other of these formats.

Originally Posted by marnold

1. Cover is displayed on Kindle for iOS (see cover_ios.jpg) but not on Kindle Voyage (see cover_kindle.jpg)

That is normal. Sending the book via Amazon's delivery system causes the KF8 portion to be delivered to a Voyage, marked as a "personal document". That combination will only display a generic cover thumbnail on an e-ink Kindle.

You should see the cover thumbnail if you instead sideload the Master MOBI to the Voyage via USB, but you then lose the ability to sync reading progress and annotations between devices.

Originally Posted by marnold

2. Page formatting is fine on Kindle Voyage (page_kindle.jpg) and Kindle Previewer (page_previewer.png), and broken on Kindle iOS (page_ios.jpg)

That is also normal. The Kindle for iOS app does not support KF8 format and so Amazon delivers the far less capable MOBI7 format to it.

Kindle for iOS supports AZK format instead of KF8. You can create that using the Kindle Previewer, but getting that file into the app can be tricky and it is not supported by any other Kindle device/app.

Originally Posted by marnold

The book is going to be distributed without KDP.
...
Any clues on how to debug/fix this?

Selling your book via the Kindle store would solve these problems. Non-purchased books are second-class citizens in the Amazon ecosystem.

[marnold]

Thanks for the reply!

A MOBI file created by KindleGen is a Master MOBI that contains both the older MOBI7 and the newer KF8 formats in a single file. Different Kindle apps/devices will use either one or the other of these formats.

I got better display results on iOS by converting epub to mobi using Calibre. Then I unpacked both mobis (kidlegen and calibre) and they all contain mobi7 and mobi8 folders. Does it mean that the preferred format is determined by some obscure factor, or simply that Calibre's MOBI7 output is better than KindleGen's one?

Selling your book via the Kindle store would solve these problems. Non-purchased books are second-class citizens in the Amazon ecosystem.

If I want to avoid the store and distribute the book for Kindle without any DRM, are MOBI/PDF formats most suitable to achieve that? I've seen some books with the AZW3 extension, can KindleGen make it?

Do I have any other options?

[j.p.s]

Originally Posted by marnold

Thanks for the reply!



I got better display results on iOS by converting epub to mobi using Calibre. Then I unpacked both mobis (kidlegen and calibre) and they all contain mobi7 and mobi8 folders. Does it mean that the preferred format is determined by some obscure factor, or simply that Calibre's MOBI7 output is better than KindleGen's one?



If I want to avoid the store and distribute the book for Kindle without any DRM, are MOBI/PDF formats most suitable to achieve that? I've seen some books with the AZW3 extension, can KindleGen make it?

Do I have any other options?

kindleunpack can extract the azw3 from the dual mobi generated by kindlegen.

https://www.mobileread.com/forums/sh...ad.php?t=61986

[jhowell]

Originally Posted by marnold

I got better display results on iOS by converting epub to mobi using Calibre.

The MOBI7 conversion from calibre may be superior to Amazon's in some cases, but may not work as well in other cases. For example, it lacks some of the data structures produced by kindlegen used for synchronization of reading position and annotations across devices.

Originally Posted by marnold

If I want to avoid the store and distribute the book for Kindle without any DRM, are MOBI/PDF formats most suitable to achieve that?

Personally I believe that a Master MOBI produced using kindlegen is the best option available for this purpose. Others may disagree.

Originally Posted by marnold

I've seen some books with the AZW3 extension, can KindleGen make it?

The AZW3 file extension used for KF8 format. KF8 is included as part of the Master MOBI produced by kindlegen.

A separate AZW3 file will be smaller than a Master MOBI, but will be far less usable since it must be sideloaded via USB to be usable on a Kindle. It will not work with "Send to Kindle".

[phillipgessert]

Originally Posted by marnold

Thanks for the reply!



I got better display results on iOS by converting epub to mobi using Calibre. Then I unpacked both mobis (kidlegen and calibre) and they all contain mobi7 and mobi8 folders. Does it mean that the preferred format is determined by some obscure factor, or simply that Calibre's MOBI7 output is better than KindleGen's one?



If I want to avoid the store and distribute the book for Kindle without any DRM, are MOBI/PDF formats most suitable to achieve that? I've seen some books with the AZW3 extension, can KindleGen make it?

Do I have any other options?

I may be mistaken, but I think the EULA for Amazon’s publishing tools prohibit sale outside of their platform (though you can give books made with them away freely). Amazon’s “publishing tools” are basically kindlegen, Previewer, and I would assume the autoconversion at the KDP dash itself. I figure this is why most off-site offerings are EPUBs with a big explainer about sideloading, the support needs of which being a big ball of wax onto itself. Being basically your question here, plus many more we couldn’t even dream of, but instead of being asked here your customers are asking them of you.

[marnold]

Originally Posted by phillipgessert

I may be mistaken, but I think the EULA for Amazon’s publishing tools prohibit sale outside of their platform (though you can give books made with them away freely).

Yep, already discovered that. I have at least one tech book that comes with a readme on how to install KindleGen and convert the epub yourself.

BTW, does anyone know how Leanpub is able to make *.mobi files that:

1. Do not violate the Amazon's EULA
2. Render okay'ish on Kindle for iOS
3. Support Enhanced Typesetting when opened with Kindle Previewer

In my experiments with KindleGen/Calibre I was unable to achieve ET with epub3 -> mobi conversion.

[Quoth]

epub2 uploaded to Amazon KDP produces Enhanced Typesetting on compatible Amazon products.

[Hitch]

Originally Posted by marnold

Yep, already discovered that. I have at least one tech book that comes with a readme on how to install KindleGen and convert the epub yourself.

BTW, does anyone know how Leanpub is able to make *.mobi files that:

1. Do not violate the Amazon's EULA
2. Render okay'ish on Kindle for iOS
3. Support Enhanced Typesetting when opened with Kindle Previewer

In my experiments with KindleGen/Calibre I was unable to achieve ET with epub3 -> mobi conversion.

If I were a betting woman, I'd guess that it's not ET, itself; it's hyphenation and you can do that with a plugin for Calibre.

The only way to make a MOBI that displays "okayish" on iOS, is by making the ebook as bloody simple as possible. iOS is not a good playground for MOBI or KF8. AZW3 won't work on iOS, either.

I mean, hell, make the files and then try to sideload them. You can only sideload "mobi" which, today, is the older, KF7 type format, (more or less).

You have bigger issues, which are the sideloading challenges that your customers will face. Amazon buyers, (and B&N buyers, Kobo buyers, etc.) are accustomed to click and MAGIC HAPPENS! The book flies through the air with the greatest of ease, and arrives on the device. PFM.

I don't know where you plan to distribute it, but if you don't spend time around "normal" people, if you are thinking of doing it yourself, you may be in for a brutal shock.

Hitch

[marnold]

Originally Posted by Hitch

You have bigger issues, which are the sideloading challenges that your customers will face. Amazon buyers, (and B&N buyers, Kobo buyers, etc.) are accustomed to click and MAGIC HAPPENS! The book flies through the air with the greatest of ease, and arrives on the device. PFM.

I don't know where you plan to distribute it, but if you don't spend time around "normal" people, if you are thinking of doing it yourself, you may be in for a brutal shock.
Hitch

This is a valid concern. However, my audience is fairly small, technical, and prefers DRM-free books. And if sideloading turns to be a huge problem, it is relatively easy to fix after the fact by publishing on Amazon.

Is there a way to glue KF7 and KF8 (generated from differently optimized epub sources) into a single mobi? Something like KindleUnpack, but in reverse (preferably through the command line).

And by the way, I figured out one way to get ET checkbox in Kindle Previewer. Pandoc-generated epub3 -> KindleGen -> mobi results in ET=off, however, Pandoc epub2 -> KindleGen -> mobi results in ET=on.

[jhowell]

Originally Posted by marnold

Is there a way to glue KF7 and KF8 (generated from differently optimized epub sources) into a single mobi? Something like KindleUnpack, but in reverse (preferably through the command line).

Nothing like that exists AFAIK. It would be difficult to do properly since a table is needed indicating corresponding parts of the two formats for synchronization and only kindlegen produces that currently.

Originally Posted by marnold

And by the way, I figured out one way to get ET checkbox in Kindle Previewer. Pandoc-generated epub3 -> KindleGen -> mobi results in ET=off, however, Pandoc epub2 -> KindleGen -> mobi results in ET=on.

The Enhanced Typesetting indicator in the Kindle Previewer indicates that the source file does not contain anything that would prevent is from being converted to KFX format which will be delivered to customers when published on Amazon.

It does not indicate that the MOBI will have Enhanced Typesetting when sideloaded. ET is a feature of KFX format.

[Hitch]

Originally Posted by jhowell

Nothing like that exists AFAIK. It would be difficult to do properly since a table is needed indicating corresponding parts of the two formats for synchronization and only kindlegen produces that currently.

Yes, I was about to say, just use KP. {shrug}. Voila, one glued-together mobi.

The Enhanced Typesetting indicator in the Kindle Previewer indicates that the source file does not contain anything that would prevent is from being converted to KFX format which will be delivered to customers when published on Amazon.

It does not indicate that the MOBI will have Enhanced Typesetting when sideloaded. ET is a feature of KFX format.[/QUOTE]


Yes, @marnold--if you are believing that ET is working, and "on," based upon the checkmark in KP3, that's very misleading. Any file you generate, via KP3, that has the checkmark, will NOT have it, when the file is viewed on a device. ET is created via the KDP publishing process--as jhowell says, it's a function of KFX, not MOBI.

As I stated previously, if you think you are seeing ET, visually when reading, in a file that came from elsewhere, that's most likely the Calibre hyphenation plugin.

With your question about leanpub, I thought you meant you were seeing it, visually, in the being-read file, on a Kindle. If you mean that you dump it on KP3 and you get a checkmark, that's no big deal. you can do that with almost any type of HTML file, ePUB2, Word...can you clarify what you meant?

Hitch

[marnold]

Originally Posted by Hitch

With your question about leanpub, I thought you meant you were seeing it, visually, in the being-read file, on a Kindle. If you mean that you dump it on KP3 and you get a checkmark, that's no big deal. you can do that with almost any type of HTML file, ePUB2, Word...can you clarify what you meant?

Yes, I saw the checkmark in KP3 and just assumed that it means ET is already enabled. Thanks for the explanation.

I think I'm going to stick with Calibre to produce a MOBI and will try to simplify the epub HTML/CSS as much as possible. If that effort will fail, then I'll resort to shipping a separately optimized PDF file produced with Pandoc.

KindleGen is a black box that requires too much experimentation. For example, just esterday I learned why it picks a wrong color for inline <code> tags in KF7 - it treats the "code span.al" CSS selector rule as "code, code span.al"...

Also, right about the time when I decided to build a Docker container with my book toolchain, Amazon discontinued KindleGen for Linux... https://news.ycombinator.com/item?id=24275231

[Doitsu]

Originally Posted by marnold

Also, right about the time when I decided to build a Docker container with my book toolchain, Amazon discontinued KindleGen for Linux... https://news.ycombinator.com/item?id=24275231

You could use Wine 5.10 to run the KindleGen binary bundled with Kindle Previewer.