Months of Runaround From KDP Support: Font Embedding Fails...HELP!

[DaleDe]

Originally Posted by Hitch

OK, so:

You mean, you could not, for example, use Arial at 1em for the body, and Arial @3em for an h1? Not to be pedantic, just clarifying your express intent.

Hitch

I would think anything not explicitly set to a font would default to the body font thus you could have Arial in more than one tag but should not explicitly set it there. That is how I would interpret the original post. Of course I have been wrong before.

Dale

[KevinH]

Hi Peter,

I examined your not-working example and can see that they did just completely strip the font out. The build.log embedded in the src shows the error was that your custom font is actually missing (can't be found!) And if you look at the embedded src archive you can see the font file is not there. So the problem is in the KDP process but it occurs before they run it through their special version of kindlegen.

So they must be unpacking and preprocessing the submitted epub in some way before processing it and when they put it back together they drop the fonts for some minor error.
If you submit a mobi, they must do the same process to its embedded source.

Have you tried submitting a mobi generated by kindlegen where you use the -dontaddsouce option? This should prevent them from preprocessing an epub and hopefully prevent the issue.

Also after reading what you wrote, I am betting, the issue is not the amount of text but which style sheet is linked-in the most often. I am guessing, whichever one is linked-in more often becomes the "main" sheet and as long as the main sheet does not have an issue and the font urls are there, embedding will be more likely. You could try subsetting all of the truly embedded fonts and urls into their own sheet and link that sheet into all documents, and moving the non-embedded font-face to it own sheet and only linking to it when necessary.
All of what I said is at least not inconsistent with what you said in your post. My theory is some preprocessing code somewhere is looking for a "main" sheet and the main sheet must be absolutely perfect otherwise pieces of it are ignored and the embedded fonts are never found.

And interestingly, a specific metadata element is added when embedded fonts are included:

528 : 'override-kindle-fonts_(528)',

When fonts are embedded this metadata item is set to "true". What is interesting here is that it literally says the the kindle device should "override" its fonts not "augment" them.

So something else to try is to also embed a Palitino font as a test to see what if any impact it has an the problem (so all named font faces with specific (non-generic) names are embedded).

If anyone knows anyone involved at Amazon, would you please check if they pre-process submitted epubs in any way before passing them to their internal kindlegen for KDP, cause its build log indicated that the font itself never made it and that was why it was "stripped-out".

I will keep looking but I don't use KDP and using kindlegen 2.9, all works as expected so I can't debug the process first-hand.

Hope something here helps.

KevinH

Originally Posted by Peter Ahlstrom

New book, new problem.

I was having the same font embedding problem with KDP as before, but the workaround was not functioning. After much trial and error (I commented out all font-family lines in the CSS and then added them back in one-by-one, then made a new CSS file with the offending line included and applied it to the html files one-by-one until it stopped working), I now have a book file where the custom font works in one half of the book but not the other half.

The offending line? Setting the font-family of the p (whether classed or unclassed) to Palatino,serif (just using the Palatino font installed on the device). If that CSS is included in the second half of the book, KDP strips out all font-family references and deletes my custom header font entirely.

"half of the book" gives a very strange effect.

• If I take the first non-working chapter and paste its entire contents into a previous chapter's file, using the Palatino font CSS in that pasted file, the font embedding works.
• If I take the first non-working chapter and duplicate it in place, then change the CSS link in the first copy of the chapter, but not the second copy, the font embedding still works. If I then delete the second copy, the font embedding now fails again.
• If I take my file and delete all the chapters after the last one that works, and upload only a half document (I made sure to also delete the now-unused images and TOC references), the fonts get stripped again.
• If I remove the CSS from the last 3 working files, and then add it to the first non-working file, the fonts embed correctly. But then if I add it to the second non-working file, it doesn't embed correctly.
• If I remove the CSS from a few more files, then add it to the last file (which is the longest html file), the embedding works. But I have to remove the CSS from exactly enough earlier files.

I also tried two methods of defining the font. One file version had the font defined in a div around all the paragraphs, and one version had the font defined for the p tag with no divs. In both cases, it works only when this is applied to one half of the book and fails as soon as that CSS is applied to one more html file after the halfway mark.

I also replaced the file completely with the book I talked about a few posts ago (a book that has 2 custom header fonts and specifies Palatino,serif for the p tag throughout)—and everything displays as expected.

This is complete nonsense. I enjoy a challenge, troubleshooting something where certain actions lead to predictable results. Here, the only predictable thing is that I can get the font to work in one half but not the other half. But only on this book, not on any previous book I've made. It's maddening.

Now, you may say "Well, just don't define the font for p." That's relatively fine on Kindle Fire and iOS, since the default font there is a serif font. But on Kindle Paperwhite the default font comes up sans-serif if I leave it undefined. I want to use sans-serif for each chapter's introductory text and then serif when the chapter proper starts. All previous Kindle books I've made have no trouble setting the p font as Palatino,serif. Why this book is suddenly different is driving me up the wall.

And yes, just converting through Kindle Previewer keeps all the fonts correctly. It's only uploading to KDP that produces the problem.

[JSWolf]

Originally Posted by Hitch

OK, so:

You mean, you could not, for example, use Arial at 1em for the body, and Arial @3em for an h1? Not to be pedantic, just clarifying your express intent.

Hitch

You just specify the font size as 3em in H1 and do not specify the font family as it will already be Arial because it was selected in the body.

[quiris]

You mean, you could not, for example, use Arial at 1em for the body, and Arial @3em for an h1? Not to be pedantic, just clarifying your express intent.

Hitch, you cannot "directly" styling using font-family property. Use CSS inheritance feature
body is ancestor of h1 in your example so you can style it according to my recipe in such way:

body {font-family: Arial; font-size: 1em}
h1 {font-size: 3em}

But according to my algorithm you cannot write:

body {font-family: Arial; font-size: 1em}
h1 {font-family: Arial; font-size: 3em}

[quiris]

Originally Posted by KevinH

Have you tried submitting a mobi generated by kindlegen where you use the -dontaddsouce option? This should prevent them from preprocessing an epub and hopefully prevent the issue.

KevinH, embedded or not embedded source epub in MOBI is irrelevant. Font stripping will be occurred in both cases.

[Hitch]

Originally Posted by quiris

Hitch, you cannot "directly" styling using font-family property. Use CSS inheritance feature
body is ancestor of h1 in your example so you can style it according to my recipe in such way:

body {font-family: Arial; font-size: 1em}
h1 {font-size: 3em}

But according to my algorithm you cannot write:

body {font-family: Arial; font-size: 1em}
h1 {font-family: Arial; font-size: 3em}

Yes. That's what I thought you meant. As we're not speaking the same native tongue, I like to clarify.

My experience with the MOBI from hell was not affected by ensuring this method of setting the heading. I'm going to pull it out this weekend, if I can, and stare at it some more, to see if I can find anything else to add to the discussion.

I can't confirm whether embedding the ePUB or not makes a difference.

I do know some folks at Amazon; but they have frankly not been helpful on this topic. As with some others, they've asked us for help--not the other way around. However, Kevin, I'll see if I can get an answer to that specific question. Assume it won't be quick. (On their parts--not that I shan't write quickly).

Hitch

[quiris]

Originally Posted by quiris

A new status update. After a couple of months waiting for answer I got another curious answer:

Here font-family "lobster" is calculated as the major font family and is placed in the body tag.

I asked KDP support a couple of weeks ago:

What factors decide which font-family is calculated as the major font-family and is placed in the body?

Today I got an answer:

Our technical team has informed me that, a font-family which constitutes to more that 50% of the content is chosen as major font-family. For this reason, font-family "lobster" is the major font-family within the content.

[KevinH]

Hi,

Originally Posted by quiris

KevinH, embedded or not embedded source epub in MOBI is irrelevant. Font stripping will be occurred in both cases.

Wow, why on earth would they strip fonts from a properly built kindlegen generated mobi with no kindlegensrc.zip added?

So they must then be unpacking the kindlegen using their own unpacking code, rebuilding an epub to result in font files being lost?

Your more than 50% rule to determine the major body font seems to be consistent with what Peter is seeming, too.

Why can't anyone from Amazon be simply given a valid kindlegen generated "working" mobi and epub, and pass it through its own process to watch the fonts being stripped and so easily reproduce the problem on their own and get it fixed.

Based on some of the posts on the web, this problem has existed literally for years!

KevinH

[Freeshadow]

Originally Posted by Hitch

Yes. That's what I thought you meant. As we're not speaking the same native tongue, I like to clarify.
Hitch

Should it ever be needed - pull me in, and we can try to solve linguistic ambiguities vie a PM 3-chain

[Hitch]

Originally Posted by KevinH

Hi,



Wow, why on earth would they strip fonts from a properly built kindlegen generated mobi with no kindlegensrc.zip added?

So they must then be unpacking the kindlegen using their own unpacking code, rebuilding an epub to result in font files being lost?

Your more than 50% rule to determine the major body font seems to be consistent with what Peter is seeming, too.

Why can't anyone from Amazon be simply given a valid kindlegen generated "working" mobi and epub, and pass it through its own process to watch the fonts being stripped and so easily reproduce the problem on their own and get it fixed.

Based on some of the posts on the web, this problem has existed literally for years!

KevinH

I don't know, but trust me: they've BEEN given those files. I've given them to them, myself. Not once, but twice. I've sent them screenshots on other occasions. Quite honestly, I'm not sure that they know. Or, if they know, they're not willing to share, for some ??? reason.

Hitch

[KevinH]

Hi,

Just for the record, here is a snippet from the not-working kindlegenbuild.log which only differs from the working version by one linked in style sheet:

        W-amzn_cap_enabled|-locale|EN|-amzncreator|dtc-epub|-tempfolder|/var/tmp/dtfc/KindlegenTempFolder/298509830|
V2.9
Linux
6
Info:I9026:option: (hidden) amazon creator tool or pipeline

<snip>


Info(prcgen):I1002: Parsing files  0000089

Warning(prcfile):W14028: Following file does not exist :   PopulaireLightEmbed.ttf

Info(cssparser):I10006: Invalid syntax for url(), ignoring the URI. "style.css"

Info(cssparser):I10006: Invalid syntax for url(), ignoring the URI. "story.css"

<snip>

So the extra font file was definitely lost in the pre-processing stage before this kindlegen version was even run (at least according to this stored log file in the KDP generated mobi).

KevinH

[Hitch]

Originally Posted by KevinH

Hi,

Just for the record, here is a snippet from the not-working kindlegenbuild.log which only differs from the working version by one linked in style sheet:

        W-amzn_cap_enabled|-locale|EN|-amzncreator|dtc-epub|-tempfolder|/var/tmp/dtfc/KindlegenTempFolder/298509830|
V2.9
Linux
6
Info:I9026:option: (hidden) amazon creator tool or pipeline

<snip>


Info(prcgen):I1002: Parsing files  0000089

Warning(prcfile):W14028: Following file does not exist :   PopulaireLightEmbed.ttf

Info(cssparser):I10006: Invalid syntax for url(), ignoring the URI. "style.css"

Info(cssparser):I10006: Invalid syntax for url(), ignoring the URI. "story.css"

<snip>

So the extra font file was definitely lost in the pre-processing stage before this kindlegen version was even run (at least according to this stored log file in the KDP generated mobi).

KevinH

Hi, Kevin:

this correlates with what we have seen; when the KDP-build fails, the
font files are stripped, in toto, from the final .mobi file. Not some, or only one; all the font files get nuked.

Is there anything I could provide to you, privately, that might help you?

Hitch

[KevinH]

Hi,
I'm afraid not. I was hoping for some additional error messages in the kindlegen build log to provide hints as to what the underlying issue really is. Unfortunately, there seems to only be the log built *after* the fonts have been stripped.

So unless KDP provides you with some additional information about what errors they are seeing it or why the fonts were stripped, we are stuck coming up with theories and then testing them via KDP runs.

I am truly surprised that submitting a kindlegen ebook without added source runs into the same problem. Is there one website or set of reference that describes the rules you and others have devised by trial and error to try to maximize the chances of success with an embedded font? There may be some overlap among them that might hold a clue. Other than that we are stuck debugging a black box. The ones that fail, remove the font so nothing useful is generated by their special kindlegen run. The ones that succeed have no problem.

Peter's test case is an interesting one in that changing just one more xhtml file from linking in one stylesheet to a very very similar stylesheet is enough to break the process, eventhough both stylesheets are valid and very nearly identical. And there does seem to be a 50% rule of some sort in play here, but it is not a max text limit, just a proportion limit. If Peter adds extra pages using the one style sheet, he can add the same number of extra pages using the other, and the fonts will stay, but if he changes just one more to use story.css instead of style.css, all fonts are stripped. So one of the differences between the two css sheets is somehow involved in this 50% rule, but I am still not sure which ones as there were a small handful of rules in style.css that did not exist in story.css. If Peter is right and it is the use of the Palatino font face, then it must in some way be related to how it is being used.

I will keep looking.

Take care,

KevinH

[KevinH]

Hi Hitch,

After re-reading the complete thread once more, I was struck by this post by quiris:

https://www.mobileread.com/forums/sho...3&postcount=67

I think Peter's case is exactly this. From the PD/KDP response, the preprocessing step is being referred to as "font-fix" tool and when it detects elements with default fonts being overwritten/changed it acts to "strip fix" all embedded fonts.

The problem seems to be when they calculate an implied "main" font based on the total text covered by that font and then implicitly apply it to all body tags, and if that results in any element that should have been a default font changed, they strip all embedded fonts. My guess is they would do the same thing if an explicitly set body font did not match the main font as measured by amount of text covered.

This I think is captured by Peter's use of Palatino on a p tag - when it covers more than 50% of the text of a book it becomes the "main" font according to the font-fix tool and therefore an implied body font, and that results in some element changing from default to Palatino, which in turn is why they throw out all embedded fonts.

So, in short, having an implied body font that somehow changes the expected font of some other element, results in all fonts being tossed.

A quick and dirty python script could in fact calculate the "main" font in a fashion similar to their "font-fix" tool to help detect this condition before submission.

Anyway, that seems to be consistent with all of the evidence in this thread related to having all embedded fonts stripped out (not just one or two flakey fonts).

Hope this helps,

KevinH

[Hitch]

Kevin:

On my iPad now, so will have to post back later, but, I can say with certainty that this is not what's happened with some of our issues. The Papyrus, certainly—that couldn't have been remotely confused with a "main" or body font. The usage was scant. Ditto the infamous Arial plus two goofy fonts debacle.

FWIW.

Hitch