Uploading a dual mobi format

[Oliva]

Hello,

Somewhere on this forum I read that if you upload to KDP a KPF file made with Kindle Create or with Kindle Previewer 3, the customers who have older KF8 devices (which are not enhanced-typesetting enabled) will receive just the old KF7 version and not the better KF8 version.

So the workaround would be to instead upload an EPUB, formatted for kf8 and kf7, and tested for enhanced typesetting compatibility.

My question is: would that work also if I upload a mobi instead, made with Kindlegen?

So the workflow would be:
1 making an epub in Sigil, with media queries for kf8 and kf7
2 testing this epub in Kindle Previewer to see if it enables Enhanced typesetting (but not converting it to kpf)
3 converting the epub from step 1 to a mobi with kindlegen using the command line
4 uploading that mobi to KDP
5 ideally, customers would receive a KF7, a KF8 or a KFX, depending on their device.

Would this work? Thank you.

[jhowell]

Originally Posted by Oliva

Somewhere on this forum I read that if you upload to KDP a KPF file made with Kindle Create or with Kindle Previewer 3, the customers who have older KF8 devices (which are not enhanced-typesetting enabled) will receive just the old KF7 version and not the better KF8 version.

I have tested this and it is true for KPF files produced using Kindle Create.

It might also be true for KPF files produced using the Kindle Previewer, but as far as I know no one has tried it. (If anyone has actually published a book using a Kindle Previewer KPF file please chime in with the ASIN and we should be able to resolve the question.)

Originally Posted by Oliva

My question is: would that work also if I upload a mobi instead, made with Kindlegen?

Yes. As long as you use the default command line options for kindlegen, especially avoiding the "-dont_append_source" option. (The included source file is needed to produce KFX - Enhanced Typesetting.)

Kindlegen is also built into the Kindle Previewer and you can export a MOBI file produced by kindlegen using the same menu choice that exports KPF. Just change the file type to MOBI in the dialog box.

[Oliva]

Thank you very much, jhowell. Very helpful.

[Quoth]

For Amazon KDP it's best to upload an epub2. They make it into old mobi, azw/kf8 and KFX with Enhanced Typesetting. This was Amazon's advice.

Unless you are making a text book or comic, it's best to work in Word or LO Writer, do a final save as docx and convert to epub2 in Calibre. Then convert epub2 to dual mobi for Smashwords only.

If you are more into layout design than writing, then you might use Sigil, InDesign (expensive), Calibre's editor etc. I can't see the point of KindleGen for anything like a novel or short story.

IMO a dual mobi has only two uses: Either to upload to Smashwords for their own Kindle downloads (also upload an epub2 as that's distributed), or to give a PD or your own text to someone with a Kindle and you don't know if it's older FW/model.

[Oliva]

Hello Quoth,
the reason i was considering uploading a mobi instead of an epub is that I cannot seem to succeed in validating the epub with epubcheck in Sigil. I always get a bunch of errors which with my very limited technical knowledge cannot understand or fix myself. I think a successful conversion to a mobi with kindlegen is a kind of verification and validation.

I use a several pieces of software for formatting ebooks. I use calibre only for some of its plugins (eg. KindleUnpack) and word processors (Atlantis, Libreoffice) only to clean up the initial text.I am also trying to integrate ebook formatting within my workflow for print. I don’t use Indesign, but free or less costly alternatives (Scribus, VivaDesigner).

Regarding the dual mobi, I’ve taken the advice of Hitch on this forum (there is so much to learn from her) and know there still are millions of KF7 devices in use, so I am trying to produce ebooks which do not look too bad on those.

[Quoth]

Originally Posted by Oliva

Hello Quoth,
the reason i was considering uploading a mobi instead of an epub is that I cannot seem to succeed in validating the epub with epubcheck in Sigil. I always get a bunch of errors which with my very limited technical knowledge cannot understand or fix myself. I think a successful conversion to a mobi with kindlegen is a kind of verification and validation.

No, Kindlegen isn't decent verification and validation.

Try a simple document first with Libre Office Writer. Edit always odt, because opening an docx messes some styles etc as it always converts.

So do a 2nd Save As before quitting Writer in docx format and try that with Sigil and Calibre.

I've been doing this for years, using paragraph styles etc. You need to manually make a TOC in Writer.
1) Make sure only items for a TOC have paragraph style of Level 1 or 2. Everything else inc. Title etc is Body Text Level.
2) Create an Index with the Insert -> Index making tool, but edit so that Tabs with the dots and page numbers are deleted and it's manual.
3) Cut the Contents page(s) and paste into any plain text editor, select and copy past back.
4) Set a suitable Paragraph style.
5) In the outline window double click on each heading in turn to go to it and add a bookmark. Do NOT select a block (that's for Word style Forms), but insert a bookmark like ch1, ch2, etc. Not full names.
6) Block select each line of content and add a hyperlink, #ch1, #ch2 etc, rather than the long winded "in the document" GUI
I have keyboard shortcuts
Ctrl A = Anchor Menu to make or delete a bookmark
Shift End will select from cursor to end of line.
Ctrl K = Edit hyperlinK (Enter bookmark prefixed with # as Internet URL without the HTTPS://)

You can put the blurb, number of words and cover info in "Properties".
Have your Author name in the User Data in Tools -> Options.
Then the first time you import the docx, Calibre will use those.
Do NOT put a cover in the Book. Those are always added separately to Calibre, Amazon or Smashwords. Set "don't detect cover" in Calibre or a first image in the book will be used.

Always save an odt before Save As docx

[Quoth]

So make an epub2. Amazon does a good job of making every Kindle format from that.
It was the ONLY way I could get uploads for old KF7, KF8 and KFX to not have "bugs".

You only need the Dual Mobi for Kindle sales on Smashwords and you need the epub2 there also, anyway.

I found I could not use a Print version for ebook generation. I find it's simpler, when the epubp2 has been proofed to death (I copy back annotations from a Kobo to PC using Kobo Utilities in Calibre) to make the formatted version for PDF for Print.

I use a small page size with no headers or footers, all pages are "default style" and style of "insert page before" isn't set. The Register and Snap are false.
For paper, I use maybe 2 to 6 page styles:
Early front matter, no page numbers
Roman numbered pages
Contents page(s)
Numbered book pages.
End matter
I use Register True (space between lines lines up through page)
Mirror margins for a small binding offset. Warning, Danger Will Robinson! Ebooks don't really have global page margins set in the book.
Headers different left & right
Footers

It can use a font like Garamond, ebooks can't.
The images (if any) need to be higher resolution and sized for the desired paper.

Mirror Danger? MS Word in particular may produce PDFs where the left & right go out of sync. Also watch out for last line of page bug. Hit return to add an extra blank line and then BS delete, otherwise at print time / PDF, the last line may fully justify instead of left justify! MS advice. Bug for maybe over 15 years?

Proof every page on screen visually of publisher's PDF for paper.

Do not print anything on paper yourself for ebooks. Save As docx, convert in calibre for epub2 and proof on a Kobo or Lithium App.

For PDF, proofing either a full HD screen or a 10" tablet, assuming no larger than 9" x 6" paper.

If you want paper, proof the epub to death, make a PDF and order a proof copy POD, I stopped printing even partial drafts on paper about 8 years ago. Last complete home printed draft novel maybe 1996.

[Quoth]

Or pay Hitch!

[Oliva]

Quoth, why do you think that KindleGen is not a good validation tool to produce a mobi? The epubs I made in Jutoh are without errors, but when I check it with epubcheck in Sigil they tend to be full of warnings. Maybe it’s Sigil’s bugs, who knows?

Of course I use styles religiously for hierarchical headings and for absolutely everything else. But I found that using Libreoffice for style formatting is ... painful. I have sometimes spent a whole day manually removing direct ad hoc formatting from some complex and long texts, before i found Atlantis word processor, Jutoh and Sigil. Same with bookmarks, toc creation, links ... There are much easier softwares that automate a lot of these tedious manual tasks, find errors, optimize the output for different destinations and much more.

Sure, simple novels are easy enough to do in a word processor. But any complex non-fiction, I don't think so. Same with font embedding and subsetting, any decorative elements, svg images, complex typography ... But maybe I'm just over-complicating.

For print, I agree, you cannot use print to make an ebook. But I found, vice versa it’s easier: i.e. if you already have a very clean ebook, it’s easy to use that to then modify or add print-specific elements in a desktop publishing software like Scribus or VivaDesigner (or InDesign). Print is its own, completely different, animal.

I’m sure with your experience you have a very efficient and streamlined workflow. I am still flailing around to find out what works for me.

[Quoth]

" have sometimes spent a whole day manually removing direct ad hoc formatting from some complex and long texts"
Only use paragraph styles. The only manual direct formatting I use is Italic for "telepathic dialogue". Everything else is named styles.

Complex non-fiction might use LaTex. Mostly it's useless as an ebook, as are large format books with photos. A PDF isn't an ebook. An ebook needs to flow for any size screen. Complex non-fiction, not on paper, needs to pretend to be paper and compatible with a 10" colour tablet as a PDF. Amazon & epub3 "fixed layouts" are really a nonsense.


This is exactly what I was saying.
"I agree, you cannot use print to make an ebook. But I found, vice versa it’s easier: i.e. if you already have a very clean ebook, it’s easy to use that to then modify or add print-specific elements in a desktop publishing software"

[DNSB]

Originally Posted by Oliva

Quoth, why do you think that KindleGen is not a good validation tool to produce a mobi? The epubs I made in Jutoh are without errors, but when I check it with epubcheck in Sigil they tend to be full of warnings. Maybe it’s Sigil’s bugs, who knows?

Umm... No. If you run epubcheck standalone, online or from the Sigil or calibre plugins, you will get the same errors. Been there, done that, have the log files to show for it.

Jutoh default error checker checks for basic structural, platform and formatting issues. To check the compiled epub for syntax and more structural errors, you need to run the bundled epubcheck and hope that it is close to being up to date. You may also want to go into preferences and click on Check epub after compiling to force epubcheck to be run.

[Hitch]

Originally Posted by Oliva

Quoth, why do you think that KindleGen is not a good validation tool to produce a mobi? The epubs I made in Jutoh are without errors, but when I check it with epubcheck in Sigil they tend to be full of warnings. Maybe it’s Sigil’s bugs, who knows?

With all due respect, we know. The folks with whom you are chatting. Unless you've done something improper with Sigil, it's not likely--not remotely likely--to be Sigil's bugs. Sigil's bugs are different and have virtually nothing to do with validation.

Of course I use styles religiously for hierarchical headings and for absolutely everything else. But I found that using Libreoffice for style formatting is ... painful. I have sometimes spent a whole day manually removing direct ad hoc formatting from some complex and long texts, before i found Atlantis word processor, Jutoh and Sigil. Same with bookmarks, toc creation, links ... There are much easier softwares that automate a lot of these tedious manual tasks, find errors, optimize the output for different destinations and much more.

I'm sorry, what software are you talking about that "automates" a lot of these tedious tasks, exactly? Anything like that--typically Apple programs--will have a godawful spaghetti of crud under the hood. Anybody here EVER seen the underside of an ePUB created with iAuthor? Moses on a pony, preserve us, it's the worst S**t I've EVER seen and kids, that's saying something. People talk about how "Word makes bad code," but Word is bloody pristine compared to the utter S***E that comes out of Pages or iAuthor. (Pages, typically; iAuthor every bloody time.) And Vellum? That magic app that all the Mac users think is the Best.Thing.Ever? Oy vey. (n.b.: to be fair to Vellum, they have obviously put in yeoman's work on the CSS and media-queries and they obviously spent a crapload of time developing "drop caps" that work even on the PPW family of devices. I gotta give props where props are due.) Or Kindle Create? Ditto, ditto, ditto. The one thing that they have in common is that they are all EASY.

Sigil is not "Word (LO/OO) for eBooks." Nobody ever said it was. Sigil is an ePUB editor and an ePUB creator, for those who know how to use HTML and CSS. It's not a WYSIWYG magic eBook maker, like Vellum, god protect us all from that POS. Sigil automates a great number of tasks--like creating the NCX and TOCs, for that matter and the OPF and I'm eternally grateful that I no longer have to create those by hand (although, absolutely everyone should, at least once, to get that knowledge in their bones.)

ATLANTIS:
I may have some guilt here. I introduced someone at KDP to Atlantis. I told him that he could use it from Word (e.g., Word-->Atlantis--ePUB) to make an ePUB, which does work complete with embedded fonts. I did warn him that the code is...well, it ain't pretty. BUT, it will get you there from here. He'd had months of fighting with Kindle Create and I simply couldn't watch it any longer. He's been spreading the gospel of AWP (and espousing my brilliance, which was an unintended consequence which I've asked him to tone down a bit), ever since. AWP should probably pay me, at this point. (j/k)

Sure, simple novels are easy enough to do in a word processor. But any complex non-fiction, I don't think so. Same with font embedding and subsetting, any decorative elements, svg images, complex typography ... But maybe I'm just over-complicating.

Weeelll, I have to confess, I personally do relatively complex non-fiction from Word my own self. For my in-house handouts and the like. It can be done. It's a bit painstaking, but the truth is, complex non-fiction in anything is painstaking. (TEX, where are you to discuss LaTEX? I mean, yes, LaTEX has plenty of non-fic muscle, but god FORFEND you then need eBooks!)

Having said that, my personal preference--yes, wait, heresy coming!--is to take a Word layout like that and kerplunk it right into INDD and then proceed forthwith to do both print and then the eBooks. IF you do it right (HUGE caveat here!), it's not horrible. YES, yes, before someone here chimes in (has you-know-who been banned again?), InDesign can create some of the world's worst cruddy code, but if you're smart, you create named styles that will nicely map to your CSS and that's half your damn fight right there. If you map styles, you don't have to clean up all that cruft later. Or at least, a lot of it.

For print, I agree, you cannot use print to make an ebook. But I found, vice versa it’s easier: i.e. if you already have a very clean ebook, it’s easy to use that to then modify or add print-specific elements in a desktop publishing software like Scribus or VivaDesigner (or InDesign). Print is its own, completely different, animal.

Dahlink, you know that I disagree with you, but with the utmost respect. Bookmaking, by and large, is more art than science, even here in the eBookWorld. P.S.: Affinity's Publisher app is on-sale, of late, for a lousy $25 buckeroos. They've been so cheap that I licensed all three--their Publisher app, their Photo app and the other one, oh, Deigner (AI's competition), each, "just in case." You never know when Adobe might do something (again) to piss me off terminally, like when they spit on those of us who'd spent many thousands on fonts and introduced their font cloud doodah. As well as, of course, spitting on those of us who'd licensed INDD for Six thou, and made it so cheap that any Tom Dick or Harriet could license it for 3 months. FIE on you, Adobe!

I’m sure with your experience you have a very efficient and streamlined workflow. I am still flailing around to find out what works for me.

And, there it is. Despite the processes and simplification (and complication) over the last decade, every eBookmaker and print bookmaker has to find their own way. Everybody has a path that they find for themselves. All the folks here can do is try to stop you from wasting a shedload of time, doing things that we all found an utter waste, ourselves, but sometimes, bookmakers need to learn things the hard way.

Hitch

[Tex2002ans]

Originally Posted by Oliva

Somewhere on this forum I read that if you upload to KDP a KPF file made with Kindle Create or with Kindle Previewer 3, the customers who have older KF8 devices (which are not enhanced-typesetting enabled) will receive just the old KF7 version and not the better KF8 version.

So the workaround would be to instead upload an EPUB, formatted for kf8 and kf7, and tested for enhanced typesetting compatibility.

Yes, using EPUB as input is usually the best way. This gives you full control over all the resulting MOBI/KF8 code.

As long as you aren't doing anything too crazy (SVG Images, MathML, etc.), you should get the "Enhanced Typesetting" green checkmark no problems. This means KFX will be generated by Amazon.

So, you use Kindle Previewer or KindleGen:

1. EPUB in.
2. Amazon's "Dual MOBI" out.

This "Dual MOBI" is what you upload to Amazon.

Note: "Dual MOBI" is a made-up term by some of us here. It's the .mobi file output by Kindle Previewer which includes 3 files inside:

• 2 Amazon-proprietary formats
  
  • MOBI (KF7) for old Kindles + KF8 for newer Kindles.
• 1 source file
  
  • EPUB, DOCX, or whatever else you used as input.

For many more details, see my 2019 post in "Epub-problems in tablets".

Note #2: Also, only relatively recently Kindle Previewer added a KPF option. Probably best to avoid this for now, because as jhowell said, we suspect that creates a MOBI (KF7) + KFX file only... completely skipping the KF8 format.

Originally Posted by Oliva

the reason i was considering uploading a mobi instead of an epub is that I cannot seem to succeed in validating the epub with epubcheck in Sigil. I always get a bunch of errors which with my very limited technical knowledge cannot understand or fix myself.

Means you have a problem inside the EPUB.

If you post some of those errors, perhaps you can get more help.

Originally Posted by Oliva

For print, I agree, you cannot use print to make an ebook. But I found, vice versa it’s easier: i.e. if you already have a very clean ebook, it’s easy to use that to then modify or add print-specific elements in a desktop publishing software like Scribus or VivaDesigner (or InDesign). Print is its own, completely different, animal.

For the most part, agreed.

But the better the source file is, the faster/cleaner/more-efficient every other future step is. You're already well on your way to achieving that by using Styles consistently.

Originally Posted by Oliva

I’m sure with your experience you have a very efficient and streamlined workflow. I am still flailing around to find out what works for me.

Yeah, as Hitch mentioned, every input->output format is going to have its own unique challenges.

Which workflow you choose also heavily depends on the skills of the person/team:

I wrote a bit about that in 2019 "Workflow for simultaneous EPUB and PDF production?"

Personally, I go "HTML-First" workflow.

I turn everything into a perfectly clean ebook, then derive all further formats from that.

Hitch (and thus BookNook) prefers "Word-First" or "InDesign-First" workflow.

Like she said, if you use Styles properly, you can use Style Mapping and other tools to keep the files in sync (very helpful if authors need to make changes to the original document).

Finalize the Print, then derive all the other formats off that clean InDesign file. Now any sort of InDesign->EPUB workflow is much faster.

Gets you to same results, just different ways of tackling the issue.

* * *

The biggest problem I see creep in is what I call the "bifurcation"—when the various formats get out of sync.

Once you split the files, you double/triple your workload if any errors/changes have to occur:

• Can you fix the equation on Page 15?
• There's a typo in "The brown cow is jumping over the moon." It should be a green cow, and the Earth.
• Can you put this image in instead?
• It should be copyright 2020, not 2019.

Once you split it into Print, EPUB/MOBI, HTML, Format X... now you have to correct it in each one. Instead of 1x amount of work, you have 3 or 4.

Originally Posted by Hitch

Sigil is not "Word (LO/OO) for eBooks."

Ugh, for my own sanity, just stop mentioning OpenOffice. It's 6+ years out of date, not updated, and withering away into dust.

LibreOffice is the true successor, and has tens of thousands of fixes/enhancements over all these years.

Side Note: If you want to see some of the stuff, check out this talk from February 2020: "Libreoffice Turns Ten and What's Next". Around 25 mins, he lists off just some of features added since the OpenOffice/LibreOffice split.

Originally Posted by Oliva

Of course I use styles religiously for hierarchical headings and for absolutely everything else. But I found that using Libreoffice for style formatting is ... painful.

Great to hear your use of Styles. You're one of the very few.

Interesting to hear issues with LibreOffice's Styles though. What's your exact pain points?

Originally Posted by Hitch

It's a bit painstaking, but the truth is, complex non-fiction in anything is painstaking.

Agreed.

Originally Posted by Hitch

You never know when Adobe might do something (again) to piss me off terminally, like when they spit on those of us who'd spent many thousands on fonts and introduced their font cloud doodah. As well as, of course, spitting on those of us who'd licensed INDD for Six thou, and made it so cheap that any Tom Dick or Harriet could license it for 3 months.

lol. Only just now they are pissing you off? They've been pissing me off forever! I refuse to touch any Adobe product with a thousand foot pole.

Side Note: And this "The Cloud" and "System as a Service" stuff is absolutely horrendous.

(One of the latest books I worked on, they used InDesign's "fantastic" Creative Cloud Stock Images... guess what, the IDML export had fully watermarked images in there... so I couldn't use those in the ebook either.)

Originally Posted by Hitch

And, there it is. Despite the processes and simplification (and complication) over the last decade, every eBookmaker and print bookmaker has to find their own way. Everybody has a path that they find for themselves.

Agreed. And there is always that unique twist that each format brings.

Sure, on the surface conversion "should be easy", but there's always these weird edge cases that creep up, or unique situations that the tools can't handle.

Fiction? Probably can be converted 100% no problem.

Non-Fiction? Most can probably be converted no problem... but there's always the next weird thing that crops up (remember footnotes-within-footnotes-within-footnotes?).

Originally Posted by Hitch

(TEX, where are you to discuss LaTEX? I mean, yes, LaTEX has plenty of non-fic muscle, but god FORFEND you then need eBooks!)

Do you want this post to turn into a tome?

(Grumble grumble, and ours was a unique case. It's possible, if only that Physics book wasn't needing insane equation alignment and unit notation. It broke all the "it should just work" tools!)

PS. But I promise, this stuff will be turned into future blog posts.

[Hitch]

Originally Posted by Tex2002ans

Hitch (and thus BookNook) prefers "Word-First" or "InDesign-First" workflow.

Like she said, if you use Styles properly, you can use Style Mapping and other tools to keep the files in sync (very helpful if authors need to make changes to the original document).

No...we do some prelim cleanup in Word if it's Word-->eBook and then we do all the major cleanup in NoteTabPro/Epsilon (HTML). We only go INDD first when we have print.

Finalize the Print, then derive all the other formats off that clean InDesign file. Now any sort of InDesign->EPUB workflow is much faster.

Yeah.

The biggest problem I see creep in is what I call the "bifurcation"—when the various formats get out of sync.

Once you split the files, you double/triple your workload if any errors/changes have to occur:

• Can you fix the equation on Page 15?
• There's a typo in "The brown cow is jumping over the moon." It should be a green cow, and the Earth.
• Can you put this image in instead?
• It should be copyright 2020, not 2019.

Yes. The moment you split into other formats, it significantly increases the work.

Once you split it into Print, EPUB/MOBI, HTML, Format X... now you have to correct it in each one. Instead of 1x amount of work, you have 3 or 4.

Yup, what he said.

Ugh, for my own sanity, just stop mentioning OpenOffice. It's 6+ years out of date, not updated, and withering away into dust.

LibreOffice is the true successor, and has tens of thousands of fixes/enhancements over all these years.

I have an OO file in my shop RIGHT now and it's a huge complex mess. Grrrr....

lol. Only just now they are pissing you off? They've been pissing me off forever! I refuse to touch any Adobe product with a thousand foot pole.

Side Note: And this "The Cloud" and "System as a Service" stuff is absolutely horrendous.

For different reasons, sugah. You have a moral objection to all paid software. I'm mad at Adobe for taking me and others like me for granted.

(One of the latest books I worked on, they used InDesign's "fantastic" Creative Cloud Stock Images... guess what, the IDML export had fully watermarked images in there... so I couldn't use those in the ebook either.)

LOL, b/c the designer didn't license them! Not Adobe's fault on that front.

Do you want this post to turn into a tome?

(Grumble grumble, and ours was a unique case. It's possible, if only that Physics book wasn't needing insane equation alignment and unit notation. It broke all the "it should just work" tools!)

PS. But I promise, this stuff will be turned into future blog posts.

We're doing another LaTEX honker right now. It's all your damned fault. HA! (Indy--that's a person--found a method that actually isn't godawful, for LaTEX->HTML. Send me a note and I'll give you what I know.)

Hitch

[Tex2002ans]

Originally Posted by Hitch

No...we do some prelim cleanup in Word if it's Word-->eBook and then we do all the major cleanup in NoteTabPro/Epsilon (HTML). We only go INDD first when we have print.

Yes, yes, should've specified.

I think it was that linked thread where we discussed InDesign-First workflows if Print was in the mix.

Originally Posted by Hitch

LOL, b/c the designer didn't license them! Not Adobe's fault on that front.

No clue. The PDF has the unwatermarked images, IDML from same person who designed them. My first time coming across it, so just let me unleash my misattributed wrath!

I don't like Adobe, I don't like them one bit!

Originally Posted by Hitch

We're doing another LaTEX honker right now. It's all your damned fault. HA! (Indy--that's a person--found a method that actually isn't godawful, for LaTEX->HTML. Send me a note and I'll give you what I know.)

Yes, yes, will do. Been too long (again, my fault).