Google Play Books and Nested Lists - Some Weird Display Issue

[Jellby]

Note that EPUB 2.1 specifies only a subset of CSS2 that's "required" by compliant rendering apps. So you may say that an app/software that doesn't support some of it is "bad", but I wouldn't dare saying anything about one that doesn't support stuff that's out of the specification. Sure, it's nice to have it, but one could blame the book as well as the app if it doesn't work (especially if the book relies on that, instead of using it as an optional enhancement).

(Disclaimer: Not talking about EPUB 3 here.)

[Quoth]

I'd regard any App that < item below> as bad:
1) can't do embedded fonts
2) ignores the justification settings in CSS
3) ignores Page Break before
4) Has no NCX
5) Links not supported
6) Ignores left, right, top or bottom paragraph css margins
7) Ignores first line indent.

Virtually every ereader ever, does all of those except embedded fonts. Only prehistoric Kindles (Before KK3 and the DXG and earlier except KK3) don't do the fonts.

Lithium and Aldiko Classic seem to work similarly to epub2 on any Kobo or Nook since at least the Touch and Simple Touch.
Some of the Binatone LCD ereaders and the iRiver Cover Story need the ebook (or Calibre created sub-directories in a specific directory, usually Digital Editions or related for fonts and CSS to work. The Boyue Likebook Mars stock reader is OK for PDFs, but KOReader is needed for decent epub support.
I think I've tested about 15 different models of dedicated ereaders and many apps on Android (4.3" to 10" screens, Android 4.x to 8.x) and two apps on the Palm PDA (which is as abysmal as you'd imagine with 160x160 pixels and older than mobi ebook formats).
Two Sony (I have 5" PRS350, tested a T1)
Three Kobo (I own two kinds now)
Maybe seven different Kindles (I own three different kinds) inc PW4 and ancient ones with KF7-mobi only.
Maybe three different LCD based Binatone eReaders, one was Android 2.x! I've only kept one as a curio.

The Kindle, Kobo and Google Books Playstore Apps for Android seem OK on Android 8.x. Our books look about the same as on a Kobo eInk or even the PW3 in Publisher mode* The Belinda Borrowbox App for our local Libraries is terrible. But the alternative is a PC download and Adobe DRM and I've not got the Adobe program on Linux, only on an ancient XP gaming machine not used on the Internet.
iRiver Cover Story (Screen cracked at rear due to battery swelling).

* Curiously the KK3 (only 167 dpi) in KF8-Publisher mode seems to render a bit more accurately like a 300 dpi Kobo than the PW3, especially with images. Oddly the PW3 makes some of them bigger rather than the same relative screen width or smaller. The KK3 even in KF7-mobi mode is better than the DXG because the KK3 has more built in fonts in FW and the stock DXG serif font is not very pretty IMO.

[Hitch]

Originally Posted by Quoth

I'd regard any App that < item below> as bad:
1) can't do embedded fonts
2) ignores the justification settings in CSS
3) ignores Page Break before
4) Has no NCX
5) Links not supported
6) Ignores left, right, top or bottom paragraph css margins
7) Ignores first line indent.

(snippage for space)
* Curiously the KK3 (only 167 dpi) in KF8-Publisher mode seems to render a bit more accurately like a 300 dpi Kobo than the PW3, especially with images. Oddly the PW3 makes some of them bigger rather than the same relative screen width or smaller. The KK3 even in KF7-mobi mode is better than the DXG because the KK3 has more built in fonts in FW and the stock DXG serif font is not very pretty IMO.

O.M.G, do not get me started on my PPW and PPW family eInk rant, about images that are meant to be larger than 50% of the width of the screen (49%, actually) and 100% of the width of the screen. I reported this problem to KDP in...2014? 2015? Foolishly, at first, I thought it was a Kindle Previewer 2.9xx problem but then I suddenly had this hunch and sure as scat, the issue exists on the real devices, too.

In all of those, if you have any image meant to be 55% of the width of the screen, KABLAMMO!, it will be 100% of the width of the screen. That's been a bug and glitch for years and years now.

Try explaining that to irate customers. (sigh). It slays me that they refuse to believe that Amazon would knowingly not fix that. "Not Amazon," they say, "it must be you. YOU must not know what you're doing."

(I have a guy right now that refuses to believe that Amazon's rendering engine won't "fix" all the issues I'm showing him, around Cambria Math. He's adamantly convinced that b/c KP3 shows it working, that despite the screenshots I've shown him, coming from my eInks, that it WILL work via ET, once it's been through the PW. He refuses to believe that the PW (publishing workflow) won't magically fix the bizarro-world view of the Cambria Math unicode characters WILL display in the eInks, which are KF8, mind you, not KFX/ET. He's certain that b/c I sideloaded the MOBIs that we produced, as tests, that that's what is causing the issue, and that Amazon's magic rendering engine/publishing workflow will MAKE the CM unicode work on all not-KF7 devices (not including KCR and/or the LookInside). [sigh and more sigh].

Hitch

[Quoth]

Originally Posted by Hitch

O.M.G, do not get me started on my PPW and PPW family eInk rant, about images that are meant to be larger than 50% of the width of the screen (49%, actually) and 100% of the width of the screen. I reported this problem to KDP in...2014? 2015? Foolishly, at first, I thought it was a Kindle Previewer 2.9xx problem but then I suddenly had this hunch and sure as scat, the issue exists on the real devices, too.

In all of those, if you have any image meant to be 55% of the width of the screen, KABLAMMO!, it will be 100% of the width of the screen. That's been a bug and glitch for years and years now.

Thanks. I was wondering was I doing something stupid, and it DID seem to depend on the image size on the PW3, yet the 167 dpi KK3 in KF8 mode seemed to work like the 300 dpi Libra and the slightly lower resolution Original H2O (227 dpi?). I was a bit baffled.

We allow maybe 2 or 3 months now between "thinking" an ebook is polished and the actual release date. So it gives time to upload epub2 to KDP and download their mysterious "preview sample". If there is something stunning new in terms of a style we also download to DXG, KK3 and PW3 from UK & USA Amazon on release and double check.
We've never had a problem with epub2 uploads (Apple, B&N, Scribd, Kobo via Smashwords and Google Books), only with KF7 or KF8.
Originally we sent "docx" to Amazon (the same one used by Calibre) and then on their conversion the page breaks vanished one day on a new ebook upload. They couldn't figure it and told us to upload whatever epub2 we send to Smashwords and Google. They didn't list epub as an upload format originally.
The Smashwords "doc" conversion is poor, so we upload epub2 and the Calibre "Dual Mobi" as Smashwords has no idea what Kindle model a customer has.

[Quoth]

Re: Epub 2.1 Spec.

I started programming with Z80 Assembler, Pascal, Occam, Forth, Modula-2 and then learned C++ BEFORE C in 1987. Then in 1988 I was introduced to the funky world of C on DOS and in late 1990s on Linux. It turns out that C originally had ambiguities and also not everyone implemented everything and not always the same way. Also there are C libraries best avoided.

Similarly real use of Word and Excel isn't a competition to use all the features. You are best with a subset that gets your job done and works on the most versions of MS Office (Used to be Office 4.3 & '95 then it was all from '97, then all from XP=2002 and later).

So I'd only be especially interested in the epub 2.1 spec if I was writing a program to render epub 2.1 files. Otherwise perhaps it whatever works on all epub dedicated ereaders (likely Adobe renderer) and seems similar on KF8 devices either converted by Amazon or Calibre. The most basic layout ought to be the same on mobi-kF7. It has to be AUTOMATIC from "odt" -> Save As "docx" and Convert to epub in Calibre without ANY editing of CSS or HTML afterwards. So not everything possible in Word or LO Styles makes sense. We do NOT try to replicate print books ever, but have a pleasant easy-to-read layout with enough but not too much whitespace.

It's best to use a clear and readible subset of HTML and CSS automatically generated from your Wordproccessor file input.

Then we edit a copy of that for paper with page styles for front matter, contents, body, appendix, rear matter etc. Maybe different body font and header spacings. We'll try then to replicate conventions of paper publishing (such as fractional spaces, headers, footers etc). I think Amazon's current idea that you can upload one file for ebook and paperback is bonkers.

We'd also only use InDesign if doing textbooks, magazines, newspapers and manuals. A crazy tool for fiction / novel paper publishing and even worse for ebooks.

[Hitch]

Originally Posted by Quoth

Thanks. I was wondering was I doing something stupid, and it DID seem to depend on the image size on the PW3, yet the 167 dpi KK3 in KF8 mode seemed to work like the 300 dpi Libra and the slightly lower resolution Original H2O (227 dpi?). I was a bit baffled.

NOPE, it's a bloody bug and they've known about it...5-6 years now. Scout's honor. I mean, hell,when was KP2.9xx last released? At least that long.

We allow maybe 2 or 3 months now between "thinking" an ebook is polished and the actual release date. So it gives time to upload epub2 to KDP and download their mysterious "preview sample". If there is something stunning new in terms of a style we also download to DXG, KK3 and PW3 from UK & USA Amazon on release and double check.

Well, I have good news and bad news for you, then. You can no longer download the mysterious "preview sample," which is nothing more than MOBI. That may be the good news, OR the bad news.

On the other hand, you can open your ePUB, HTML, whatever, in KP3, and then use file-->Export and get a MOBI for sideloading, as well as an AZK file (for sideloading to iOS devices). Ta-da! See, I'm a problem-solving wench for you today, Quoth. Again, that may be the good, or bad, news. Not really sure...

We've never had a problem with epub2 uploads (Apple, B&N, Scribd, Kobo via Smashwords and Google Books), only with KF7 or KF8.

Ehhh, I've had my moments with B&N. I'll never forget finding out the hard way that if you used a span inside the first paragraph of a chapter head that you were screwed. (For incipits).

Originally we sent "docx" to Amazon (the same one used by Calibre) and then on their conversion the page breaks vanished one day on a new ebook upload. They couldn't figure it and told us to upload whatever epub2 we send to Smashwords and Google. They didn't list epub as an upload format originally.
The Smashwords "doc" conversion is poor, so we upload epub2 and the Calibre "Dual Mobi" as Smashwords has no idea what Kindle model a customer has.

Well, a looong time ago and far far away, when the earth was young...and Amazon was betwixt and tween K1 and K2 and were working on the next gen, you could just use docx, because effectively, everything was being converted by MOBIPocket Creator. Hell, I learned originally, pre_K2, on MBPC.

But the moment that they starting moving away from that and wrote their own rendering machine, yes, you had to move up to zipped HTML or ePUB to get reliable, consistent results. I think I switched up to ePUB, full-time, in early 2010.

Hitch

[MicroDrie]

Originally Posted by RbnJrg

The best ereader apps (regarding as how they display an epub) are:

1) Reasily
2) Gitden
3) PocketBook
4) Lithium
5) Overdrive

The difference between 1-3 are tiny; I use mainly PocketBook because it has features not present in Reasily or Gitden (for example, Reasily doesn't have TTS nor hyphens). But the support for css3 in ereaders 1-3 is very good (to my surprise, they even support calc and custom variables).

Instead of buying an EPUB reader, I invested a long time ago in a smart-phone and tablet with the free GitDen EPUB reader. If you want to know your EPUB reader can or can't, if you have an EPUB reader with JavaScript you can use the attached EPUBtestHTML.epub to see what your E-reader can and can't.

Back to the question of this post, it is clear that the manufacturers of E-readers are not in much haste to update their equipment with recent and less bugged software.

Considering the foregoing, the question remains, what can we recommend rsuchwani?

I think there is no good, but unfortunately only the least bad solution. Actually, we can't find any real flaws in rsuchwani EPUB.

Today, almost everyone has a smartphone in their pocket. I would therefore opt for a mobile first approach with a re-flowable layout and a larger font. In addition to validating the EPUB, I would test it via a good EPUB app on a smartphone or tablet. For those who are fond of a book layout, I would print the EPUB to a PDF format with the current font size.

I think the latter, printing to PDF, is the crux of the problem. When I added an EPUB extension in both Chrome and Firefox, rsuchwani EPUB was not showing properly. I missed pieces of text, or the text didn't stick to the bottom margin.

If I print a page to PDF from Sigil with the Microsoft print to PDF printer, it works flawlessly. The disadvantage is of course that you have to print and merge each page separately.

The question remains for rsuchwani how was the PDF created, and how did you see the errors in the EPUB?

[Hitch]

Originally Posted by MicroDrie

Instead of buying an EPUB reader, I invested a long time ago in a smart-phone and tablet with the free GitDen EPUB reader. If you want to know your EPUB reader can or can't, if you have an EPUB reader with JavaScript you can use the attached EPUBtestHTML.epub to see what your E-reader can and can't.

No, all you know is what your ePUB-reading software can and can't. You don't know bupkus about what the devices themselves can do. Case in point--I just mentioned here, in a thread, about spans inside the first-paragraph style on B&N Nook readers. Worked perfectly in ADE and any number of software devices and crashed and burned like an out-of-fuel drone on a real Nook.

You can't know jack about how files work on devices, without the devices themselves. That's why so many of us in the business own a plethora of devices.

Back to the question of this post, it is clear that the manufacturers of E-readers are not in much haste to update their equipment with recent and less bugged software.

Or even not-so-recently updated software.

Considering the foregoing, the question remains, what can we recommend rsuchwani?

We've all made our recommendations, most of which basically said that you're going to have to make sacrifices one way or the other--either the graphics, or the distribution plans.

Today, almost everyone has a smartphone in their pocket. I would therefore opt for a mobile first approach with a re-flowable layout and a larger font. In addition to validating the EPUB, I would test it via a good EPUB app on a smartphone or tablet. For those who are fond of a book layout, I would print the EPUB to a PDF format with the current font size.

Well, that's one way to go, but of course, when you go to distribute the book(s), generally, the big retailers DO have associated devices.

I think the latter, printing to PDF, is the crux of the problem. When I added an EPUB extension in both Chrome and Firefox, rsuchwani EPUB was not showing properly. I missed pieces of text, or the text didn't stick to the bottom margin.

If I print a page to PDF from Sigil with the Microsoft print to PDF printer, it works flawlessly. The disadvantage is of course that you have to print and merge each page separately.

Not to mention that the regular, every-day reader of eBooks wouldn't know what Sigil is, how to print-to-PDF or any such thing. I deal with regular, everyday, not-MobileRead people every bloody day and the normal, everyday citizen knows how to a) browse the net (and MIGHT know what "a browser" is) and b) how to look at their email.

80% of the people I deal with do not know what downloading is, or how to download from a browser; they think that "downloading" means clicking an email attachment twice. Oh and the ones that don't use file extensions on their computers (both Mac and PC)? Don't get me started.

There is NO universe in which thinking that buyers or readers will load an ePUB up, in Sigil, and print-to-PDF is a remotely viable solution. I'm sorry but unless his reading demographic happens to be eBook nerds, that's just...it's downright silliness. Won't happen and you're just torturing the buyers--and more importantly, yourself--if you try to get them to do that.

If you're going to do PDFs, for the love of Mike, just distribute PDFs and stop all the faffing around and twaffling.

The question remains for rsuchwani how was the PDF created, and how did you see the errors in the EPUB?

If you say so.

Hitch

[MicroDrie]

Dear rsuchwani, from the fact that you have attached a PDF to your question, I infer that you have a serious PDF problem. You can answer my question in a private email to me. You can find the private mail option by clicking on MicroDrie at the top left of this post. I will then see how you can help in the simplest way.

[rsuchwani]

I've been silently going through the experts' comments and hoping to get some 'hurrah' moment to get rid of the issue I have. You guys have given so much time to this thread and sharing so much knowledge that I'm truly overwhelmed and for that I deeply and heartily thank everybody.

Back to the topic at hand, @MicroDrie, the semi-solution (having readers print/export to PDF) that you are providing is not going to work because it just doesn't seem right to even suggest to the author of the book and all the readers. In this regard, @Hitch is right to turn down that suggestion.

A little bit of background, the author of the book is now a retired pediatrician who has written this book over the years putting every bit of advice he can for the parents of young babies. Each list indicates some kind of 'procedure to follow' in case of any health-related concerns, thus specialized list markers. It was a long process to get everything right for his liking and there were times that each involved got frustrated with each other. @Hitch would know what I'm talking about.

Anyway, we have only targeted 2 apps/readers (iBooks and Google Books) at the moment and if we get positive responses and sales, we'll approach other markets as well. As you all may know, iBooks has a very decent SDK and can handle everything you through at it so the ePub doesn't have any display issue there, but Google Books has only these 'weird empty alpha, beta-level list markers on the following page, but only with nested lists, other list items/markers displays fine'.

I know that if the ePub is browsed on other apps/readers it'll have issues (due to the different reader engine, etc.) but I'm not concerned about that at the moment, my only concern is to be able to resolve this GPB issue.

One more thing, @MicroDrie,

The question remains for rsuchwani how was the PDF created, and how did you see the errors in the EPUB?

I don't think that I fully get what you exactly want to know here. If you're talking about the PDF I've attached to my question, then it is just a collection of screenshots I've taken of the ePub on my Android phone indicating the issues I have. I have only created ePub from the start.

[Quoth]

Originally Posted by Hitch

You can't know jack about how files work on devices, without the devices themselves. That's why so many of us in the business own a plethora of devices.

Yes. Absolutely. It's also why on Android and the Mars Eink I have the Kindle and Kobo Apps as well as KOReader.
I did give my Nook Simple Touch to my grandson, but if I do something new I can check it on that. If I don't edit any of my existing tested LO Writer styles and don't create new ones then I don't need to retest a new ebook made by docx-> Calibre import -> Convert to epub2 with default settings.
I think about 1/2 of readers use an App (including Amazon Fire which is an App on an Amazon version of an Android Tablet) . After that Kindle eink could be nearly 90%, Kobo next, then B&N Nook. Not sure of proportion of Tolino (Android versions of Kobo, similar HW & same owner). Sony and Binatone are now only old ereaders, they've not made one for years; the Sony electronic paper doesn't count. Do iRiver still make a reader?
After that you have the niche Boyue, Oynx, Pocketbook etc most of which are customised Android using third party Android Apps.

Amazon has about 90% of the English ebook sales, though more than 1/2 of our sales are via Smashwords. I guess many publishers and self Publishers only use Amazon? Amazon has about 80% of online paper book sales in USA and dominates English language audio-books, but they are mostly listened to on phones.
Android has about 82% of world wide phone/tablets, but USA has very much higher percentage of iOS. But Google's ebook sales are tiny; it's not what people use the Playstore for. I don't know relative proportion of Amazon Kindle App vs Apple Books use on iOS. We've sold a similar number to Apple, Kobo and Barnes & Noble (Nook) via Smashwords. Hardly any to Tolino or Scribd via Smashwords.

Unless you are doing textbooks, manuals or comics the ebooks should be as simple as possible. Test WP styles on real devices (Old Kindle with KF7 AND KF8 on 2016 or later Kindles) and Kobo Touch or later as a minimum.

Test on Android on 4.3", 6", 7" and 10" screens. At least Android 4.x and 8.x with sensible Apps.

When I did have Amazon PC tools and preview on Windows I found them pointless. The Calibre Viewer allows a quick check that all page breaks, headings, links and system index is correct (NCX) and styles seem OK.

Two models of Kindle seems enough, though I have three and access to four other Kindles. One model of Kobo seems enough, though I have two. The screen failed on the Kobo Touch.
The 5" Sony PRS-350, a 4.3" LCD Android 4.x, 6" Android phone, 7" and 10" Android tablet, Amazon KK3, PW3 and Kobo Libra check image formats & sizes make sense. I only check basic layout and readability on the DXG with its horrid tiny selection of stock fonts. Amazon trimmed costs too much on DX and DXG by having too little RAM.

If I'm fixing downloaded PD or bought books I usually only check on the Calibre viewer and mostly delete stuff in the CSS after a Calibre conversion. i don't test like I do when a new or edited wordprocessor style is used.

I don't like embedded marginalia original page numbers in fiction. Some downloaded PD. I use regex search & replace on the HTML to delete them in Calibre Editor. If the formatting and OCR proofing is REALLY bad I'll convert to RTF, properly style in LO Writer, using odt native editing and a final Save As in docx for Calibre. Some of the old PD texts in Amazon Megapacks seem to be badly done from Internet Archive scans with OCR. Like all the Gothic novels Jane Austin mentions in Northanger Abbey. Yes, those are all real and some quite fun. I suspect while JA was poking fun she might have also been a fan!

[Quoth]

If an epub:
1) Passes both the epub checkers in Calibre Editor
2) Looks OK on Calibre Viewer (Index, headings, page breaks, links) in a phone size window.
3) Passes Smashwords test after direct epup2 upload and is accepted for premium catalogue.

THEN
a) It's going to be on Apple Book, Kobo eink & App, B&N Nook, Tolino and Scribed.
b) There should be zero issues uploading the IDENTICAL epub2 to Amazon KDP for Kindle devices, Fire Tablet and Kindle App. Amazon has 90% of English language ebook market
c) There should be zero issues uploading the IDENTICAL epub2 to Goggle Books for the playstore. But we have never sold a single copy via Google and their ebook share is tiny.

You do not need to do anything special for Google. You do not need an Apple version and SDK if you distribute to Apple via Smashwords.

Though you SHOULD test also on at least a 167 dpi Kindle, 300 dpi Kindle and a Kobo (the Libra is the best features/value). The Libra is also brilliant for proof reading, annotation then export via Calibre to a PC/MAC text file beside the WP source on screen. I stopped proofing on paper years ago when I got a Kindle. But the Kobo is better.


PDFs are primarily for creating paper print, or for use on approximately 7" or better screens (if about A5 or smaller, not A4 or Letter) when the content has tables or formula or requirements that genuinely don't work on a 167 dpi 6" eink (epub2, KF8 and KF7).

PDFs are terrible as an ebook solution.

[MicroDrie]

Originally Posted by rsuchwani

Google Books has only these 'weird empty alpha, beta-level list markers on the following page, but only with nested lists, other list items/markers displays fine'.

I've already looked at the Google Books terms and conditions. They don't allow JavaScript (isn't in it) and prefer EPUB 3.0.1 specification. I can't find the reason there either. Now what I'm curious about is the difference in rendering of the original EPUB before sending to Google Books, downloaded again from Google Books and read again in the same EPUB reader and rendered EPUB. Only if there is a difference with the same EPUB reader, we have established that something goes wrong during processing at Google.

[Quoth]

Real ebooks don't have javascript. It's only needed for Interactive.

Google are totally fine with epub2.1 and AFAIK only do anything to the ebook if you ask for DRM. DRM is evil and doesn't stop commercial pirates. Only limits regular users and creates nasty walled gardens.

If you can't do a good ebook in epub 2.1 that seems nearly identical on KF8 Kindle then either you are doing it wrong or should use a framework and create iOS and Android apps. Certainly anything more than static text + static images needs an app. The epub3 and HTML5 is too limited and also awkward to distribute as you don't know how compatible any epub3 app or Browser might be.

If something seems to go wrong at Google, then you are doing something to clever or wrong.
Run the two checkers in the Calibre Editor.
Upload epub2 to Smashwords and see if it meets standard for Premium Catalogue.
Upload epub 2 to Amazon KDP and see if it passes.
You can set release date nearly a year away on both and then cancel (gives a bad rep) or unpublish later if you want.

Google is practically the least fussy, though they do check. They'll spot invalid ISBNs. I've never seen evidence of Google changing the ebook, but I don't know how they add DRM. Likely that just encrypts it without changing anything, but we NEVER request DRM, so I don't really know. The ONLY thing ever I've seen with Google is complaints about the metadata.

[MicroDrie]

Originally Posted by Quoth

If something seems to go wrong at Google, then you are doing something to clever or wrong.

There's nothing wrong with the PlayBooks_NestedLists_Test_V03.epub file as long as you don't read it with an extension in Chrome or Firefox. Those extensions have a problem. That's why I want to rule out that the E-reader is not the problem.