(Solution) Formatting issues converting from azw3 to epub

[SillyStegosaurus]

Posting so if someone is looking for a solution to this problem I had, they can find this. I was not able to find an answer by searching, but I figured it out.

I was having trouble converting an azw3 to epub in Calibre. I tried a few different screen formats -- generic e-ink, generic e-ink HD, even Kindle) and in all cases, the conversion was cutting off some of the letters on the left side of some right pages. This showed up even when I opened it in an epub reader on my computer, confirming it wasn't an issue with the ereader, it was a file issue.

The fix: I converted the azw3 to a mobi, confirmed there were no missing letters, deleted the azw3, and then converted the mobi to an epub. After that it was formatted properly.

[DNSB]

Did you try using the KindleUnpack plugin as an alternative to calibre's conversion?

[Quoth]

Originally Posted by SillyStegosaurus

The fix: I converted the azw3 to a mobi, confirmed there were no missing letters, deleted the azw3, and then converted the mobi to an epub. After that it was formatted properly.

That's a bad fix. Something is wrong with a conversion setting. Try kindle unpack, because mobi throws away formatting etc.

The Page dialog is for image resizing. Pick Tablet and images are not resized.

It's a margin setting is the issue. Maybe the original has negative margins because the Kindle adds margin. That can be fixed either with kindle unpack or looking at the CSS (of azw3 or converted epub). All epub viewers/eink can put zero at the edge, so any negative margin to reduce the added margin on a real kindle will chop off parts.

[SillyStegosaurus]

Thank you for the suggestions! How would I go about fixing it with Kindle Unpack? I installed the plugin, unpacked the book, and noticed some oddities. Books.app said the epub was corrupted and could not be opened. Calibre's epub reader opened it up, but it retained the formatting issue where some pages were cut off at the margins. I put it on my Kobo and the poor device froze outright when I tried to move past page 2 of the book! I had to connect to my computer and delete the book to get it working again.

Or, in the convert screen, when I set the page layout to "tablet", it replicated the bug. Similarly when I set the margins to 0 to try preserving the original layout.

[Quoth]

Tablet only affects the images.

You need to look at the ebook in the Editor and open the style file (css). Setting the Margin =0 in conversion may only affect what Calibre considers to be body text.

Check there are no -values in the CSS.

[SillyStegosaurus]

The entirety of the az3w's css is pretty sparse, and it has no negative values.

Posted below:

#data data="
@page Section1 { size:612pt 792pt; margin:36pt 72pt 72pt }
div.Section1 { page:Section1 }
">#data>

Those margins seem pretty big. I note that there are only 3 values in the margin, not 4, but there should be 4 margins right? (I'm not very familiar with css or html)

[Quoth]

Delete that.

Yes, there are four margins but there are various shorthand ways. I usually explicitly have four.

[JSWolf]

I really dislike the margin shorthand in CSS. I prefer margins to be shown the long way with each margin on one line.

[DNSB]

Originally Posted by SillyStegosaurus

The entirety of the az3w's css is pretty sparse, and it has no negative values.

Posted below:

#data data="
@page Section1 { size:612pt 792pt; margin:36pt 72pt 72pt }
div.Section1 { page:Section1 }
">#data>

Those margins seem pretty big. I note that there are only 3 values in the margin, not 4, but there should be 4 margins right? (I'm not very familiar with css or html)

That would indicate the book's creator used inline styles instead of CSS classes. Generally those are a mess to clean up. I generally start with Sigil's RemoveInlineStyles to convert them to a CSS stylesheet and go from there.

For what it's worth, that margin shorthand is top, left and right, bottom.

For a quick and dirty start, I would remove the @Page directive.

[SillyStegosaurus]

Alright, I installed the RemoveInlineStyles plugin to Sigil and opened an epub Calibre made from the original az3w.

"This EPUB has HTML files that are not well formed or are missing a DOCTYPE, html, head or body elements. Sigil can automatically fix these files, although this may very rarely result in minor data loss in extreme circumstances.

Do you want to automatically fix the files?"

I had it automatically fix the files, then applied the plugin to it. According to the output, no inline styles were found at all!

Checked the epub's stylesheet.css:

Spoiler:

".calibre {
display: block;
font-size: 1em;
padding-left: 0;
padding-right: 0;
margin: 0
}
.calibre1 {
display: block
}
.calibre2 {
display: block;
margin-bottom: 0%;
margin-top: 0%;
orphans: 0;
text-align: center;
widows: 0
}
.calibre3 {
-aw-left-pos: 46.45pt;
-aw-rel-hpos: column;
-aw-rel-vpos: paragraph;
-aw-top-pos: 0;
-aw-wrap-type: inline;
height: auto;
width: 80.29%
}
.calibre4 {
display: block
}
.calibre5 {
display: block;
margin-bottom: 0%;
margin-top: 0%;
orphans: 0;
text-align: center;
widows: 0
}
.calibre6 {
-aw-left-pos: 18pt;
-aw-rel-hpos: column;
-aw-rel-vpos: paragraph;
-aw-top-pos: 18.6pt;
-aw-wrap-type: inline;
height: auto;
width: 92.31%
}
.calibre7 {
-aw-left-pos: 76.7pt;
-aw-rel-hpos: column;
-aw-rel-vpos: paragraph;
-aw-top-pos: 0;
-aw-wrap-type: inline;
height: auto;
width: 67.31%
}
.calibre8 {
font-size: 0.75em
}
.calibre9 {
font-size: 0.75em;
text-decoration: none
}
.calibre10 {
font-size: 0.75em;
font-style: normal;
font-variant: normal;
text-transform: none
}

"

Not really sure how to make sense of this. The epub I got from converting azw3 -> mobi -> epub is so far the only readable epub.

[DNSB]

Originally Posted by SillyStegosaurus

Alright, I installed the RemoveInlineStyles plugin to Sigil and opened an epub Calibre made from the original az3w.

"This EPUB has HTML files that are not well formed or are missing a DOCTYPE, html, head or body elements. Sigil can automatically fix these files, although this may very rarely result in minor data loss in extreme circumstances.

Do you want to automatically fix the files?"

I had it automatically fix the files, then applied the plugin to it. According to the output, no inline styles were found at all!

Checked the epub's stylesheet.css:

".calibre {
display: block;
font-size: 1em;
padding-left: 0;
padding-right: 0;
margin: 0
}
... lines deleted

Not really sure how to make sense of this. The epub I got from converting azw3 -> mobi -> epub is so far the only readable epub.

Basically, use KindleUnpack to extract the epub and then edit that. What you have above is a calibre conversion which converts inline styles to CSS classes but tends to muck around with the structure of the epub and also, creates epubs that are not standards compliant. Kovid Goyal (author of calibre) will argue that those standards have bits that are not needed/outdated and epubcheck is anal but many publishers will reject to reject the ebook when epubcheck does not pass it and more than a few ereaders and apps will have issues.

[SillyStegosaurus]

KindleUnpacked to get an epub.
Opened in Sigil. Selected "no" when it asked if I wanted Sigil to automatically fix defects.
Applied RemoveInlineStyles plugin.
Plugin error: Incorrect XHTML/XML Detected. `Incorrect XHTML: OEBPS/Text/part0000.xhtml Line/Col 18,49 Entity 'nbsp' not declared.`

The Styles/OEBPS/Styles/stylesheet.css is full of this string, "&nbsp", all over the place.

I had it continue anyway to see what would happen. Output:
`Status: success

No inline styles found in: Text/cover_page.xhtml
Removed inline styles from: Text/part0000.xhtml
Inline styles converted to new stylesheet: OEBPS/Styles/sgc_styles.css
`

Saved, hit "no" to "fix on save". Result: Corrupt and unopenable book.
Saved another copy and let it fix on save. Result: Corrupt and unopenable book.

Repeated both of the above, but this time I manually edited all references to the original stylesheet to point to the new stylesheet. Result: Book would open and then Books.app freezes. Or with Calibre's epub reader, the text was still cut off on the left side.

Repeated process but let it do the fix first. Similar results.

Thank you for the help but I think at this point I should just stop burning time trying to get a perfect conversion and go with the good-enough albeit slightly unstyled version I made at the time of my first post. I'd like to know how to fix this properly, but I feel like we're getting into "do a discord call and screenshare to pair program in html" territory.

[DNSB]

After saying no to the automatically fix, I would suggest under the Tools menu, click on Reformat HTML and then Mend and Prettify All HTML Files. Basically, it will attempt to fix the easily fixable errors. This should clean up the error you got from the plugin.

OTOH, it sounds as if this ebook is one of the idiocies where all content is stuffed into a single xhtml file.

And a question that I likely should have asked earlier is the title/author of the book. It's possible that someone could grab a sample and see just how screwed up it is.

[JSWolf]

It sounds to me like this eBook would be best converted to ePub with calibre. Do not use KindleUnpack. Let calibre change the inline styles to CSS and to split any HTML files that are too long.

[Quoth]

&nbsp is just a non-breaking space. Those should be rare.