Mobipocket vs ePub: Why worse is better in ebook formats

[wallcraft]

If MobiPocket was still an independent company I think it would be busy extending the MOBI format to allow it to display all (or most) of the ePub format. This would not fix the "don't render the markup correctly" issue, since this is the entire point of the MobiPocket approach, but it would allow MOBI to compete on a more or less level playing field going forward.

One way of looking at ePub is that it is a single target for publishers. This is how it is apparently viewed in the UK for example. An extended MOBI format does not break this approach, because ePub is the source but on the device you get MOBI. Just as today, the source is an OEB 1.0 ebook and the target is MOBI.

Another way of looking at ePub as at the reading device level, with the argument that reading devices are getting more powerful all the time and so can render ePub directly. Mobile ADE is the only current implementation of that approach. If FictionWise actually switches from eReader to ePub it will be (or might be) the 2nd. Note that other "ePub" software on handhelds is not even close to conforming to the standard (I'm not sure about Stanza, but I think it is in the largely non-conforming category). In most cases, if a small-screen device also supports MOBI then using Calibre for ePub to MOBI first will give a better result on the screen.

[DaleDe]

Originally Posted by wallcraft

If MobiPocket was still an independent company I think it would be busy extending the MOBI format to allow it to display all (or most) of the ePub format. This would not fix the "don't render the markup correctly" issue, since this is the entire point of the MobiPocket approach, but it would allow MOBI to compete on a more or less level playing field going forward.

One way of looking at ePub is that it is a single target for publishers. This is how it is apparently viewed in the UK for example. An extended MOBI format does not break this approach, because ePub is the source but on the device you get MOBI. Just as today, the source is an OEB 1.0 ebook and the target is MOBI.

Another way of looking at ePub as at the reading device level, with the argument that reading devices are getting more powerful all the time and so can render ePub directly. Mobile ADE is the only current implementation of that approach. If FictionWise actually switches from eReader to ePub it will be (or might be) the 2nd. Note that other "ePub" software on handhelds is not even close to conforming to the standard (I'm not sure about Stanza, but I think it is in the largely non-conforming category). In most cases, if a small-screen device also supports MOBI then using Calibre for ePub to MOBI first will give a better result on the screen.

Mobipocket already has a converter for ePUB as the source format and does even support some CSS in the creation process but removes it in the final format. So it can use ePUB as a publisher source although I don't think it is currently as good as it could be.

As a reading device Adobe is not the only game in town although it may have the best rendering currently. Hanlin can read ePUB and FBReader can also. FBReader is better than Hanlin but both are available. ZuluReader for Windows Mobile devices is coming along nicely these days. I agree that converting to Mobi is much better on a Hanlin device than reading ePUB.

I agree with the original author on most of what he says but towards the end when he says Mobi displays wrong but still displays. This is a function of the Reader itself and not the format of the data. This trick would work with almost any format if you simply ignored the format and rendered the data until you could sync up.

His comments on Lit are interesting. Hanlin can render LIT but the performance is terrible. Now I better understand why. I actually like LIT for its rendering ability and built in image viewing on later readers. As all the eBook Readers move to 400 MegaHertz processors this will be improved I believe. The new power management can allow this without giving up too much battery life.

CSS can be unbearably complicated with all the multi-levels and overrides. This is the reality of CSS and makes rendering difficult. Faster processors will help but some better coding guidelines I think would help even more. It needn't be as complicated as it is for eBooks.

Dale

[HarryT]

Originally Posted by DaleDe

I agree with the original author on most of what he says but towards the end when he says Mobi displays wrong but still displays. This is a function of the Reader itself and not the format of the data.

It is in a sense an issue with the "data format", Dale. If you jump to an arbitrary point in a Mobi file you have no way of knowing what current "state" you should be in - bold, italic, centred, left-justified, fully-justified, or anything else. That's why ePub loads the entire XML parse-tree into memory, so it has valid state data for every "node" in the tree and knows how to render it "correctly" if you jump arbitrarily to that node.

This "problem" with Mobi is precisely what allows it to work "well enough" on ANY device, no matter how limited its memory resources may be, since you're only ever dealing with a single "page" of the book at a time.

[DaleDe]

Originally Posted by HarryT

It is in a sense an issue with the "data format", Dale. If you jump to an arbitrary point in a Mobi file you have no way of knowing what current "state" you should be in - bold, italic, centred, left-justified, fully-justified, or anything else. That's why ePub loads the entire XML parse-tree into memory, so it has valid state data for every "node" in the tree and knows how to render it "correctly" if you jump arbitrarily to that node.

This "problem" with Mobi is precisely what allows it to work "well enough" on ANY device, no matter how limited its memory resources may be, since you're only ever dealing with a single "page" of the book at a time.

Yea, and I suspect ePUB could be rendered without getting the context right either if you wanted to settle for that. It would be readable but not right. This is the same as MOBI and is an HTML by product, not unique to mobi. In ePUB the big complication is that you first have to read the CSS and set it up before rendering anything but, having done that, you are likely to be correct or almost correct even if you jump in the middle (depending on how the file is formatted of course which is why I say it needs guidelines).

By the way the mobi format has no notion of a page, nada, none.

Dale

[HarryT]

Originally Posted by DaleDe

Yea, and I suspect ePUB could be rendered without getting the context right either if you wanted to settle for that. It would be readable but not right.

Yes. That is, I imagine, what the ePub programs for mobile phones, PDAs, etc, actually true.

By the way the mobi format has no notion of a page, nada, none.

True, but the reading application does.

[tompe]

Originally Posted by DaleDe

I agree with the original author on most of what he says but towards the end when he says Mobi displays wrong but still displays. This is a function of the Reader itself and not the format of the data. This trick would work with almost any format if you simply ignored the format and rendered the data until you could sync up.

Well it depends. A format can also specify what is conformant rendering. Also if it is specified in the format different allowed ways to render things then the tools generating the files can take that into account to try to avoid error in the rendering. So I really think that the intended use of the format belongs to the format specification.

[netseeker]

Originally Posted by HarryT

That's why ePub loads the entire XML parse-tree into memory, so it has valid state data for every "node" in the tree and knows how to render it "correctly" if you jump arbitrarily to that node.

Can mobile browsers render even large XHTML-documents correctly? Yes. Do they need to load the whole document into memory? No. ePub-renderers also don't need to do that. It's just a matter of how the software developers implement their CSS renderer and not really a matter of "XML parse-trees".
Even on mobile devices XML can be used as stream. The parsing can be done using event models (SAX) or newer approaches (StaX). The XML isn't the problem, some CSS-styles are. Mobile browsers were able to solve that and hence ePub-renderers could solve that too.

[Nate the great]

Originally Posted by HarryT

Yes. That is, I imagine, what the ePub programs for mobile phones, PDAs, etc, actually true.



True, but the reading application does.

Not that I can tell. If you jump to a link in the middle of a book, when you click previous page the response will be erratic. It won't display the closest text. Instead it will jump some amount further towards the front of the book and display from there.

[HarryT]

Sorry, Nate, are you talking about Mobi or ePub?

[Nate the great]

Originally Posted by HarryT

Sorry, Nate, are you talking about Mobi or ePub?

I thought we were talking about Mobi.

[HarryT]

We're comparing ePub and Mobi, hence my slight confusion.

My experience with Mobi is that when you jump via a hyperlink, the point jumped to will always be displayed at the very top of the page. Going backwards through a Mobi file is always a rather "hit and miss" affair, and very often doesn't exactly "match up" with what you get paging forward through the file.

[kovidgoyal]

Just to confirm that the author of that article really is llasram (whom I haven't heard from in a while).

It is certainly true that MOBI is currently a more "stable" platform for which to create ebooks. It's much easier to take random HTML and convert it to a MOBI e-book that will work than an EPUB e-book that will work. The price you pay for that is lowered fidelity.

However, I think the problem with EPUB is simply that of growing pains. Supporting CSS 2.1 means that EPUB renderers must load entire XHTML flows. This in turn means that EPUB creation software must impose some sort of limit on XHTML file sizes. As people get used to that, not working EPUB files will become less common. After EPUB is still very young.

[netseeker]

Originally Posted by kovidgoyal

However, I think the problem with EPUB is simply that of growing pains. Supporting CSS 2.1 means that EPUB renderers must load entire XHTML flows.

No, that's just not true, kovid. You need just a basic knowledge about the parent tree of the element you are trying to render. That's a huge difference to "entire XHTML flows". Stream the xhtml, store the information about the parent tree that's really required into memory and "forget" the rest.

[HarryT]

Originally Posted by netseeker

No, that's just not true, kovid. You need just a basic knowledge about the parent tree of the element you are trying to render. That's a huge difference to "entire XHTML flows". Stream the xhtml, store the information about the parent tree that's really required into memory and "forget" the rest.

But wouldn't that imply that, in order to jump to any point in the file, you'd need to "stream" all the file content up to the point you're jumping to? That could be extremely slow, couldn't it? I think that's the whole point of parsing the file once and then storing the parse tree in memory, isn't it?

[netseeker]

Originally Posted by HarryT

Are you suggesting that, if you wanted to jump to a point 9.9MB into a 10MB file, you'd need to "stream" all the preceding 9.9MB of the file in order to get the state data for the point you're jumping to? That sounds horribly time consuming. I think that's the whole point of parsing the file once and then storing the parse tree in memory, isn't it?

You just need segments of the parse tree in memory to apply the required styles. So you can overead large parts of the 9.9 MB. But yes, for more distant positions in a big file you would have to read/inspect large parts of the file (again). That's what some mobile browsers are doing. Look, mobile device these days have powerful CPUs and very fast I/O. It's just the way it works. You can't hold the complete parse tree in memory on all devices.