What exactly are Standard Ebook's "advanced epubs"?

[murraypaul]

Originally Posted by JSWolf

So please stop say you cannot support ADE 2.0.1. You can if you wanted to.

They don't want to.
Kindles don't need it.
Kobos don't need it, if using KePub.
One does not need to satisfy all possible users, nor is that normally a good use of time.

[JSWolf]

I am fixing up the eBook I mentioned. I will post it here once I am done and prove what compatible is.

[acabal]

Originally Posted by JSWolf

I am fixing up the eBook I mentioned. I will post it here once I am done and prove what compatible is.

I'm sure you can craft an epub that would work on any particular reading system you choose; so can we, and that's not the point. The point is that we aren't targeting 8 year old ebook renderers, just like modern websites don't target Netscape Navigator or IE10. I'm sorry you don't like that, but that's the decision we have to make as a volunteer organization with limited time and resources.

We get to release one main ebook file--the compatible one--and it has to work reasonably well on as many modern--emphasis on modern--systems as possible. ADE 2.0.1 is not in that list because it's ancient, but systems like Kobo, Kindle, Play Books, iBooks, Bluefire, and so on, are.

It would be absurd to demand for a website released today to work on IE10, a web browser released in 2012, for it to be deemed "compatible". The same can be said for epubs, which are just websites-in-a-box that are rendered on what boils down to glorified web browsers. Any web developer who was working during the IE6 days will tell you how hard, and even impossible, it was to craft a single file that would work reasonably well across several browsers/renderers, let alone catering to demands from the peanut gallery to support obsolete pet favorites too.

Edit: I'll restate my invitation for you to volunteer on working on our build chain to build in the compatibility you desire. The source is on GitHub and anyone can create a pull request or discuss real technical solutions. But, note that we expect a professional standard of conduct.

[JSWolf]

Here is the compatible ePub. You already have the ePub 3 version so there really isn't any need for as the compatible ePub because it's not really compatible. Have a look at this and see how simple the CSS and code is and how compatible it is. It's not complex and it works with RMSDK. RMSDK is the most widely used software for reading ePub so your eBooks really should work with it.

[JSWolf]

Originally Posted by acabal

I'm sure you can craft an epub that would work on any particular reading system you choose; so can we, and that's not the point. The point is that we aren't targeting 8 year old ebook renderers, just like modern websites don't target Netscape Navigator or IE10. I'm sorry you don't like that, but that's the decision we have to make as a volunteer organization with limited time and resources.

We get to release one main ebook file--the compatible one--and it has to work reasonably well on as many modern--emphasis on modern--systems as possible. ADE 2.0.1 is not in that list because it's ancient, but systems like Kobo, Kindle, Play Books, iBooks, Bluefire, and so on, are.

It would be absurd to demand for a website released today to work on IE10, a web browser released in 2012, for it to be deemed "compatible". The same can be said for epubs, which are just websites-in-a-box that are rendered on what boils down to glorified web browsers. Any web developer who was working during the IE6 days will tell you how hard, and even impossible, it was to craft a single file that would work reasonably well across several browsers/renderers, let alone catering to demands from the peanut gallery to support obsolete pet favorites too.

Edit: I'll restate my invitation for you to volunteer on working on our build chain to build in the compatibility you desire. The source is on GitHub and anyone can create a pull request or discuss real technical solutions. But, note that we expect a professional standard of conduct.

Speaking of Bluefire, does McTeague by Frank Norris display correctly? Bluefire is built on a slightly newer version of RMSDK. I believe it's one RMSDK versions used in ADE 3.x.

Thing is, if you can code a website that looks like you want and does what you want but can be made compatible with more web browsers, why not do so especially if the code is no more complex then the way it was originally planned to be coded? It just makes more sense to have the most comparability possible. In the case of eBooks, the code is a lot simpler and it will now work on many more programs/Readers. You could take the version I posted, convert it to ePub 3 and still have it work with RMSDK. The most changes would have to be done in the OPF. But the CSS would not need to be changed. The HTML would not need to be changed all that much unless something needs fixing to be ePub 3 compatible.

[PeterT]

Jon; the thing you need to understand is that SE have tools to automate the creation of their offerings. I've not looked at what you've done but the ideal would be to leave the (x)html files alone and just supply a new stylesheet that is ADE 2.0.1 compliant.

[JSWolf]

Originally Posted by PeterT

Jon; the thing you need to understand is that SE have tools to automate the creation of their offerings. I've not looked at what you've done but the ideal would be to leave the (x)html files alone and just supply a new stylesheet that is ADE 2.0.1 compliant.

I'll have a go at replacing the CSS from the original and seeing what happens with ADE 2.0.1. I'll have to see what classes would need to go or be changed. I don't think it's much.

[acabal]

Originally Posted by JSWolf

Here is the compatible ePub. You already have the ePub 3 version so there really isn't any need for as the compatible ePub because it's not really compatible. Have a look at this and see how simple the CSS and code is and how compatible it is. It's not complex and it works with RMSDK. RMSDK is the most widely used software for reading ePub so your eBooks really should work with it.

With this ebook you haven't done anything that our compatible epub build doesn't already do. You use CSS class selectors, just like our compatible epub build does. Our compatible build also uses > and + selectors which IIRC were supported on older ADE versions anyway.

However you downgraded to epub 2.0.1--the version of the spec from 2007--while our compatible epub is epub 3.2; you removed all epub:type attributes, which are important for accessibility (screen readers for the visually impaired, and so on); and you stripped almost all metadata. I consider all of those big losses. Besides that, there is no obvious technical difference between the two, unless I missed something. (Which is entirely possible, there are too many differences here for a folder diff to be meaningful.)

I appreciate what you're trying to do, even though I disagree that you've succeeded in demonstrating anything. But you're straining to keep ebooks trapped in an IE6 world. The web, and ereading systems, have moved on.

Arguing in this thread is going to be largely pointless. If you have real technical solutions, I invite you to open a GitHub discussion or even a pull request to implement them yourself. Here is a link to our toolset: https://github.com/standardebooks/tools

[JSWolf]

Originally Posted by acabal

With this ebook you haven't done anything that our compatible epub build doesn't already do. You use CSS class selectors, just like our compatible epub build does. Our compatible build also uses > and + selectors which IIRC were supported on older ADE versions anyway.

However you downgraded to epub 2.0.1--the version of the spec from 2007--while our compatible epub is epub 3.2; you removed all epub:type attributes, which are important for accessibility (screen readers for the visually impaired, and so on); and you stripped almost all metadata. I consider all of those big losses. Besides that, there is no obvious technical difference between the two, unless I missed something. (Which is entirely possible, there are too many differences here for a folder diff to be meaningful.)

I appreciate what you're trying to do, even though I disagree that you've succeeded in demonstrating anything. But you're straining to keep ebooks trapped in an IE6 world. The web, and ereading systems, have moved on.

Arguing in this thread is going to be largely pointless. If you have real technical solutions, I invite you to open a GitHub discussion or even a pull request to implement them yourself. Here is a link to our toolset: https://github.com/standardebooks/tools

The reason to do what I did is maximum comparability with the most software. You have the advanced version which has all of what I removed. The problem is that the compatible version isn't very compatible.But my changes make it much more compatible. So you would have the best of both worlds. You would have a version that most can read and you'd have a version that's your ePub 3.2 advanced version.

The problem as I see it is that sure, you are using the ePub 3.2 spec, but most software for reading ePub barely handles ePub 3.0 let alone ePub 3.2. So because of this, you are doing a lot of work that a lot of people won't be able to read because it won't work or it won't look good.

[acabal]

Originally Posted by JSWolf

The problem as I see it is that sure, you are using the ePub 3.2 spec, but most software for reading ePub barely handles ePub 3.0 let alone ePub 3.2. So because of this, you are doing a lot of work that a lot of people won't be able to read because it won't work or it won't look good.

epub 3.2 is backwards-compatible with epub 3.0.1. Reading systems that can do one can generally do the other. I agree that adoption of epub 3.0 was slow, but the epub 3.0 spec was released in 2011, and the world has had 9 years to build compatible ereaders. Today I consider adoption of the 3.0 spec to be at an acceptable level, and we must dig our heels in and move forward instead of forever catering to the past. After all, if we were really serious about backwards compatibility, we should do what Project Gutenberg does and simply release plain text ebooks and forget epub all together. After all, are we really compatible if an ebook can't be read on an MS-DOS terminal from 1985?

Fortunately, all of the ebooks we produce at Standard Ebooks are structured consistently, because part of the point of the project is to create ebooks that are easy to machine process. If you wanted to read one of our books on your epub 2.x reader, you could write a script to do the conversion, and the script would in theory work across our entire corpus.

[davidfor]

Originally Posted by acabal

SE Editor-in-Chief here.

ADE is the IE6 of ereaders. There isn't a whole lot we can do to make things look nice there. Fortunately fewer and fewer devices are shipping with ADE. Kindle of course uses their own renderer for mobi/azw3, and Kobo uses a very good Webkit-based renderer when using kepub files (but importantly, it defaults to ADE when using plain epub, versus kepub, files--that's why we tell people to transfer the kepub files to Kobo readers and not the regular epubs, otherwise ADE will make the ebooks look bad!).

Firstly, we need to make a distinction. "ADE" is the software running on PCs and MACs for handling ePubs especially those with Adobe DRM. The "Adobe RMSDK" is what is used on many ereaders and apps when reading epubs. The distinction is important as they don't necessarily render books in the same way, and the versions of each that someone see is not likely to match. I would never suggest anyone use ADE for anything other than books infested with DRM and maybe as a quick check of rendering.

Having said that...

Sorry, but that is BS. The Adobe RMSDK is just as good at rendering books as the kepub renderer is. I use both. And I would have to look closely to tell the difference. Or check the book format. The RMSDK does not support some newer features, but, in general there is no differences in how they look. There is a lot of discussion about kepubs and epubs on Kobo devices. Which to choose really comes down to personal opinion. And whether people can be bothered installing the software to change books to kepubs. There are differences in the font handling and epub does win out there. For example, the Adobe RMSDK will use ligatures, whereas the kepub renderer won't. And for an ugly error in a kepub, just look for a em-dash in a justified book. Some people think these things are important. Some don't care. Some people don't want kepubs and want to stick with epubs. In my opinion, the only real advantages using kepubs over the epub are the way that footnotes are handled, the ability to view images with zoom and the in-book stats. If you are reading novels, only the last one will be

The reason that your "compatible" epubs look terrible on a Kobo device is because of something you are doing. Something in the stylesheet is causing an error in the RMSDK and that dumps the complete stylesheet. I have had a look and what I believe is wrong is at the end of this post.

There is also a problem with recommending kepubs: Kobo does not officially support sideloading kepubs. While it works, the only reference they have to this is in their guide to authors. They make a suggestion for how to test book looks OK on Kobo devices. And the only other instructions are about them are how to make sure links, the ToC and footnotes work properly after Kobo does their processing when the book is submitted to their store. This means that at any point, Kobo could remove this support. And if someone finds a bug that cannot be reproduced in a purchased kepub, it might not be fixed (and yes, I know of one). I don't think Kobo will remove the support, but, it won't surprise me if it happens. I'll just assume they have a new manager that wants to make their mark. Or it will be some idiot in marketing.

And the problem is that your "compatible" epubs aren't. They are not compatible with the version of the Adobe RMSDK that is in the current firmware for Kobo devices. Or tolino ereaders. Which means they are even worse for older devices. I would have expected Nooks to have similar problems. Can you tell me what ereaders they are compatible with?

And the fact that these are not compatible with these older devices means that you are alienating a group of people that might need these books. There are still plenty of people using these old devices. I saw a post today from someone who was happy to receive a Kobo WiFi for Christmas. That is a device that hasn't been made for 9 years. And that backs up your point about how slowly the ereader technology has moved. The companies got this working very well years ago. Unless you break a device, there is little reason to upgrade. And that is also why the newer ePub standards were not adopted. They were just not needed.

And for the record, the mention by others of ADE version 2.01, is a little bit of a red herring. There are two reasons it is still recommended here. One is to help demonstrate a book is compatible with older devices. Such as Sony's or the older Kobo devices that do not get firmware updates. Plus quite a few ereader applications. But the big reason is that books downloaded with it can have the DRM removed. This isn't guaranteed with the later versions of ADE.

Now, how to fix the "compatible" books. And honestly, the "Advanced" books.
And for the record, the bit of the stylesheet that is breaking the books in the RMSDK on Kobo devices is:

@media (prefers-color-scheme: dark){
	img.epub-type-se-image-color-depth-black-on-transparent{
		filter: invert(100%);
	}
}

This is the first time I can remember seeing an @media statement in the stylesheet for and ebook that didn't have the test for the media type. Most have been for Kindle specific things and I think they all used "screen" (of course I can't find them now). Changing this to the following works. It should work with "screen", but, "all" is effectively what I think your current code is doing.

@media all and (prefers-color-scheme: dark){
	img.epub-type-se-image-color-depth-black-on-transparent{
		filter: invert(100%);
	}
}

Whether you make this change or not is up to you. But, at the moment, your claim of "Compatible epub" is not particularly valid.

And add "widows:1;orphans:1" to the body. Anyone using epubs on Kobo or Sony device

[Uncle Robin]

the only real advantages using kepubs over the epub are the way that footnotes are handled, the ability to view images with zoom and the in-book stats. If you are reading novels, only the last one will be

I have read MANY novels with images that benefited from zooming in, and more than a few with footnotes one outstanding example with plenty of both is The Count of Monte Cristo - pretty sure that's considered a novel. All generalizations are dangerous.

[JSWolf]

Originally Posted by Uncle Robin

I have read MANY novels with images that benefited from zooming in, and more than a few with footnotes one outstanding example with plenty of both is The Count of Monte Cristo - pretty sure that's considered a novel. All generalizations are dangerous.

Most novels do not have pictures or footnotes. I know some do and it you want to read the footnotes or be able to zoom the graphics, yes, KePub is better. But some books I've recently read that had footnotes I did not need t use a single footnotes as the footnotes were references to citations that I was not going to read. I don't need to know what publication some bit of the book was taken from. But be that it may, the problem is that these eBooks do not work with RMSDK.

If you look at Sony, nook, and Pocketbook, they all use some version RMSDK. Not everyone installs KOReader or even knows about it. So why insist on code that doesn't work?

I tested with @Davidfor's fix and it did not work, but removing that media query did allow the eBook to mostly work. But it still needs fixing because not all formatting working. But the code does not need to be so complicated I formatted the eBook with a lot simpler CSS code. And it's much more compatible.

[robin_reala]

Of the 435 books currently in the SE corpus, 176 (or 40%) have endnotes.

[davidfor]

Originally Posted by Uncle Robin

I have read MANY novels with images that benefited from zooming in, and more than a few with footnotes one outstanding example with plenty of both is The Count of Monte Cristo - pretty sure that's considered a novel. All generalizations are dangerous.

Originally Posted by robin_reala

Of the 435 books currently in the SE corpus, 176 (or 40%) have endnotes.

Honestly, that is what you pick up on in my post? Pretty much a throw-away remark that I probably should have qualified. Great, your experience is that. In my library, there is probably less than ten novels that have any footnotes. And the only ones that have any images in them are basically fantasy novels with a map. And guess what, when I read those, I loaded them on my Kobo ereader as kepubs so I could look at the map a little easier. The only novel I can think of that made me do this because of footnotes was "World War Z" by Max Brooks.

But, I'll stand my statement that for novels, footnotes and images are relatively rare. For classics, the situation is different. And I'm willing to put money down that there have been extremely few novels that have been on the best-sellers list in the last 50 years that had footnotes and multiple images in them.

And that isn't the issue. The issue it that the "Compatible epubs" are broken on the most common epub ereader out there (yes, I am sure of that). And also broken on the other epub ereader I have (a tolino epos). And, that @acabal seems to be of the opinion that "ADE will make the ebooks look bad!". Which is a problem for two reasons. ADE is not involved, it is the Adobe RMSDK (yes a little pedantic, but they are not the same thing) and, from my experience, the differences in rendering between the different renderers are rarely significant. So, please tell me how the RMSDK makes the books look bad.