PDF is not an eBook format

[ahi]

Originally Posted by frabjous

We can, and should, have both reflow and decent typography!

Or, simpler still, both reflow and fixed layouts.

- Ahi

[frabjous]

Originally Posted by DawnFalcon

No, you don't need to use a DVI driver, but any other means of rendering is going to have higher overheads. It's what DVI is designed for, after all. In the meantime, you're saying that the PDF output works? Well yea, but that dosn't deal with my requirement, for instance, for multiple screen sizes without using different documents and losing my annotations.

I don't see why not. The above are all created with the same document, and attaching annotations would not be difficult. The overhead issues with PDF output would not significant long-term (and indeed barely exist now; my experience is that PS or PDF files are about 30% larger than the corresponding DVI files), and while I don't really know how hard it would be to produce a DVI driver for ARM (I doubt very), it's probably the sort of thing that no one has even attempted, but might if there were a reason to.

I look forward to when ePub renderers can do MathML. I've looked at MathML somewhat, and while it fits the bill with this sort of thing in the sense that it would give a decent look, it is seriously bloated compared to TeX, and would be nightmare to actually compose in. In fact, I think right now most people write in TeX and convert to MathML. It's one of the only ways to actually produce MathML.

But in principle I have no qualms about MathML; my arguments have been in favor of a better renderer; I don't care if that renderer uses TeX source or some kind of (X)(HT)(Math)ML, as I've explained multiple times -- it just needs to be improved to the same level as is already possible with TeX. (I was the first person in this thread to mention MathML.) My problem has never been with ePub. Indeed, I've said multiple times that I have hopes that we can get this kind of quality rendering with ePub. The only thing I've suggested is that, firstly, until we get a decent renderer for ePub, PDF is still a worthwhile format. And seconly, I've suggested that since TeX exists as an alternative source format, where just about all the tools we'd need are already in place, there's no reason to set our expectations low: if corporate publishers don't want to switch to it, then they owe us something as good (which could certainly take the form of extended ePub mark-up with a better renderer...)

Putting both fixed and reflowable formats into the same document is a so-so compromise, but it really seems unnecessary, since just about all the tools needed for a typographically superior reflowable single document already exist.

[DawnFalcon]

Well, you have to remember that PDF annotation is all third-party programs, and they don't tend to play nice. You can easily synch ePuB annotations.

Yes, it'd be relatively easy, if time consuming, to write a DVI driver for ARM-based CPU's, but there isn't one yet. Who's going to write it? Remember that it's going to add to, rather than remove market fragmentation, and ePuB has only just been chosen by Sony, Google, etc. to consolidate arround (and importantly, it's a standard, so they can't be accused of anti-trust or locking people out).

I have to disagree on PDF's overhead thought - it is anything but insignificant. Not every reader out there has a SD slot, and even ePuB has a lower page count than LRX on the Sony's because it's less efficient - PDF's are even less so, by a larger margin. Minimal rendering was not one of the PDF design goals, and neither was reflow - PDF reflow is absolutely and totally dire at present, and Adobe don't seem to have any plans to address it, prefering to advance ePuB.

We're back to Better Is Worse. Applying that to ebook formatting makes a lot of sense, where PDF's effectively apply the MIT Approach.

And, um, it's pretty trivial to write MathML graphically. Indeed, I've never written it any other way. It's also used in a large number of apps these days which have nothing to do with TeX whatsoever.

[frabjous]

Once again, I'm not arguing against ePub. The things I'm saying about TeX is just to get a sense of what's already possible as a way of gauging what reasonable expectations are with regard to ebooks. No one need write a DVI driver for ARM if they provide something analogous with whatever they want to use instead.

I think you misunderstood what I was suggesting about annotations too. The annotations wouldn't go into the PDF, they'd go into the source document (which could be an ePub-like TeX-bundle or whatever). The renderer would make a PDF out of the source and display that. It would only need to have one PDF at a time, so the greater amount of space of a PDF as opposed to a DVI or something else would only apply to the active document, and how likely is that to be very significant? In any case, if current specs aren't there, they will be soon. And again, I'm not really pushing for this. It's just that we're pretty much almost there already, so there's no excuse for not getting something as good for what they're actually selling us.

Writing things graphically only works if you're using common notation, or very sophisticated software the likes of which I haven't seen. My background is in philosophy, not math, and most of the "math" I'm interested is actually logic, including arcane notations from the history of logic. No graphical composition engine is going to know what I'm trying to write out of th ebox, not to mention that I don't have any suitable implements with which to do it. (You really want me to draw on my screen with a mouse?) Introducting a new TeX command to type is trivial. But again, I don't object to MathML--if it becomes standard, the tools will come.

[DawnFalcon]

Yea, I'm just pointing out that PDF dosn't currently have a standard way of handling annotations (which is really, really annoying, I'd add).

And have you ever tried Mathematica? (which exports to MathML)

[frabjous]

Again, my background isn't math. I'd have very little use for Mathematica... and, unlike TeX, it isn't free. But I'm glad it exports to MathML.

[WillAdams]

Actually, .pdfs have had a standard way to handle annotations since v4 when Adobe licensed a 3rd party system for doing this --- unfortunately, the best solution for this, comment-enabled .pdfs (made using Acrobat v8 or later's Comments | Enable for Commenting and Analysis in Adobe Reader menu command) forces the .pdf to be encrypted.

William

[PhishStyx]

I just love that this is 31 pages, literally thousands (perhaps 10's of thousands) of words, and basically it's a half dozen people all talking past each about the definition of "e-book."

[Sonist]

Originally Posted by PhishStyx

I just love that this is 31 pages, literally thousands (perhaps 10's of thousands) of words, and basically it's a half dozen people all talking past each about the definition of "e-book."

But of course, some of us are sensible, and the others are..., well..., birdbrains.

[Tuna]

Originally Posted by frabjous

Once again, I'm not arguing against ePub. The things I'm saying about TeX is just to get a sense of what's already possible as a way of gauging what reasonable expectations are with regard to ebooks. No one need write a DVI driver for ARM if they provide something analogous with whatever they want to use instead.

It's probably important to separate out the various features and abilities of the formats and renderers being discussed. From the point of view of ebooks, the 'headline feature' of TeX is the combination of a mature and highly evolved renderer and fonts that have been analysed for the best possible layout and spacing rules. Neither of those are inherent in the document format - they're features of the software.

TeX does of course allow much more than this, but which of those other features are a priority for ebooks? Why have the manufacturers chosen PDF and ePub over TeX?

In general, PDF has a lot of disadvantages for ebooks - it obfuscates the information required to do reflow, it has issues of legacy support, it's 'expensive' to render and it's a proprietary format. Most of the discussions in this thread have focussed on over-complex work arounds for the reflow issues which at least acknowledges that the future of e-readers will encompass a wide range of screen sizes. Rather than trying to fix a format that is out of our control and inherently incapable of reflow, wouldn't it make more sense to improve a format that can do reflow but is weak in the area of rendering hints?

There's a strong argument for a community effort to develop a 'better' epub renderer as an open source project. It's quite widely recognised that the existing renderers are inconsistent and do a less than optimal job. Again, that's not something inherent in the format - it's the software that is letting the reader down. If such a renderer were developed, it could be used by future readers, and could also be used as a lever by which the community could push the evolution of the format. As ePub has become the de facto alternative to PDF, it would be better to work with it than attempt to introduce another format. Applying the TeX reflow and rendering rules to ePub could produce a dramatically improved experience and open the path to extending ePub to support further rendering hints and advanced layouts for things like formulae.

The point is that epub could be improved by extensions that allow for more complex layout and rendering. The format itself can be extended and renderers can be designed to degrade sensibly where extensions are not supported in the software. PDF however is resistant to reflow and extension and is likely to remain so for many of the reasons discussed.

Other formats like TeX are interesting for showing what is possible, but not really an option for e-readers now that ePub is becoming standard. The trick is picking out which benefits of other formats such as TeX are down to software, and which are down to the format itself.

[WillAdams]

There are several standards for .pdf which are ISO standards, and the technical specification for the format is readily available, so it's not what I would call proprietary --- lots of different implementations as well, several of which are free and opensource.

Similarly, while the TeX source code has a rather interesting license, it's widely accepted in the industry that it is acceptable to create commercial, closed source implementations based on it --- and there're several successors which have more open licenses.

As regards the whole ``PDF is not an eBook format'' --- it devolves down to the old joke, ``How many legs does a dog have if you call a tail a leg?''.

Just because a format doesn't suit a particular person's usage for a particular set of formats, doesn't mean that a format isn't part of that set.

On the bright side, at least no one has launched an ``ePub files are not eBooks, but eScrolls'' thread.

William

[Abecedary]

Originally Posted by WillAdams

On the bright side, at least no one has launched an ``ePub files are not eBooks, but eScrolls'' thread.

Don't worry. I guarantee there will be one by the end of the day. And does this mean we can start demanding that they bring the Scroll Lock key back for our readers?

[jbjb]

Originally Posted by ahi

Because typography is not a machine-solvable problem.

You've made this assertion several times, but I've yet to see your justification for it. (Note that "not machine-solvable" != "not machine-solved yet").

/JB

[ahi]

Originally Posted by WillAdams

Just because a format doesn't suit a particular person's usage for a particular set of formats, doesn't mean that a format isn't part of that set.

If we typography zealots didn't eat babies and didn't want to deny people with eye-conditions the ability to read, you'd be right.

- Ahi

[ahi]

Originally Posted by jbjb

You've made this assertion several times, but I've yet to see your justification for it. (Note that "not machine-solvable" != "not machine-solved yet").

/JB

Keep, reading, I guess. (Note that "not machine-solved yet" != "machine-solvable".)

- Ahi

Ps.: Or why don't I help you out...

And what I am saying, and will continue to state--what with it being the plain truth and all--is that no reflow format, particularly not one based on HTML, can do a good job with typography. Why? Because trying to do so hands part of the typographic work off to software... and typography is not a software-solvable problem.

The easiest to understand issue is hyphenation. If software cannot reliable and accurately hyphenate basically all words contained in a document, it has every chance of encountering situations where it may be able to make no typographically sound decision.

In English, ignoring the fact that there are well over a million words, which may be combined in impossible to foresee ways to produce, for all intents and purposes, infinitely more words... not to mention the possibility of foreign words being included in the text, proper names whether common or esoteric. If the software cannot figure out the correct hyphenation pattern for all the words in a given eBook, it cannot arbitrarily reflow and expect to be able to render something still professional-looking. (I suppose only reading eBooks about working class Americans, with common Christian names, going to well-known popular places, doing things everybody knows about/has done before is a good way to minimise software hyphenation problems.)

In other languages, you can easily get into a situation where a given word can mean two (or more) entirely different things and therefore also have differing correct hyphenation based on what they actually mean in the given context... context that may be difficult to impossible to determine via software due to the language's relatively free word order. Not that the idea that eBook reading software should have to know anything about the semantics of the text isn't ludicrous to begin with... because it truly is, and there are languages where without that, getting hyphenation right is literally hopeless. Unless of course you will have a human being spend more time manually pre-soft-hyphenating the entire document than it would take to typeset it to popular eBook reading device screen sizes as tagged PDFs (so they reflow to less popular eBook reading device sizes in the less than optimal way they cannot help but do).