PDF is not an eBook format

[Sonist]

Originally Posted by Lemurion

... As the population ages, the ability to select a larger or thicker font will become more and more important for readers who don't need the finest examples of typography, but do need to be able to see the words.

You may prefer PDF, they can't use it.

I don't understand the point.

Take a non-image PDF and open it on your PC. Now, zoom in. The text stays nice and crisp.

Similarly, the Adobe Mobile SDK 9 reflows simple text, and allows enlarging the font. But it also can deal with complex, fixed layouts, where required.

So, what exactly is the problem?

I can see the validity of kovidgoyal's "semantic" argument (although I don't see why it can't be largely addressed by future development of PDF,) but don't see the issue here.

[thibaulthalpern]

Originally Posted by JSWolf

thibaulthalpern please have a look at the attachment I am linking. It is a sample ePub that I'd like you to try to find fault with.

https://www.mobileread.com/forums/att...0&d=1240007547

Thanks for sharing. Right from the outset I can tell you that there is kerning problem in the text block that begins "Department of Electrical and Computer Engineering" which is under the name David Akopian.

Scanning down, more kerning problems in the text block beginning "Correspondence should be addressed to David..."

And then immediately scanning quickly down to the section labelled "Introduction" more kerning problems. You'll notice some lines are really squished in while others have disproportionate amount of white space between letters and between words.

It's quite an ugly piece of formatted text. Nay, it's not really formatted at all because there is no kerning and professional typographical capabilities in this type of digital format, as far as I understand.

[thibaulthalpern]

Originally Posted by kovidgoyal

By semantic, I mean that information in the source document is marked up by type rather than by position/appearance. See for example the new tags in HTML 5 (the next iteration of HTML). http://en.wikipedia.org/wiki/HTML_5

Note that semanticity does not preclude fixed size layouts. My point is simply that the more important aspect of information distribution is making the distribution more semantic, rather than better laid out. The advantages to making the information moe semantic far outweigh the advantages to making it better laid out. Given that, a format that is designed primarily to facilitate layout is unlikely to be the future of digital information.

If I could add to this, I think by semantic Kovid means markup language. In other words, similar to metadata. Semantic in his usage is defining blocks of text in categories.

So, one may define x-block of text as "header", y-block of text as "body text", a z-block of text as "caption", or even finer gradations you have a-block of text marked in category "font: Times New Roman" and so forth.

This is how I understand what is meant by semantic.

[thibaulthalpern]

Originally Posted by Sonist

I still disagree with the position, that "the advantages to making the information more semantic far outweigh the advantages to making it better laid out."

This is an old, old problem. Anyone who is knowledgeable or a professional in graphic design and typography please chime in. I claim to be no expert but I do claim to have worked in it in the past and have also take a course or two on graphic design and typography as part of my undergraduate major (quite a long time back now).

When digital type emerged (as opposed to metal typefaces), there were many problems that computers could not at that time solve. Ligatures was one problem. An example of a ligature is below. Notice how when you type
"fi"
in certain font typefaces, if you don't have advanced ligature capabilities, the dot of the "i" runs into the hook of the "f". This is demonstrated in the top right of the image below.

With advanced ligature capabilities, the "fi" becomes a glyph (see image, top left)which I obviously cannot show you textually because this feature is not available in this forum nor is it available generally on the web. So, I demonstrate again with the graphic below. Notice how in proper ligature format, the "dot" of the "i" has disappeared. In fact, it no longer exists! The hook of the "f" takes over the space that originally would have been inhabited by the "dot" of the "i".



The other example given is
"fl".
Notice that without advanced ligatures, the top serif of the l runs into the top serif of the f making it a total jumbled mess.

With advanced ligature capabilities "fl" is blended as a glyph (i.e., one character). Now, before advanced ligatures (which only came around early 2000s, I believe, if my memory serves me correctly) the computer has no concept of ligatures.) You had to manually find all incidences where there were ligature problems and replace the problem with a glyph if your font actually had that glyph. Most fonts don't actually have those glyphs built in. Now, with advanced ligatures (a feature that actually is part of the font and part of the text program you use and most still actually DON'T have them) this problem is solved because when I type "fi" or "fl" or "ffl" etc. in say Adobe InDesign using an advanced font typeface the "fi" is automatically replaced by the proper looking "fi" and the computer can still understand this as "fi" and not a glyph (because a glyph is one character so "fi" in the past when properly ligatured would become a single entity and thus if you search for "fi" as "f" and "i" you won't find the word!!!

Now, this only touches the surface of typographical problems. There are many problems of which mentioned previously by someone else was kerning. Automatic kerning still isn't good enough to be used totally professionally. If you want a properly kerned text, you really still need to do it by eyesight.

And the problem goes on and on and on. If a piece of text was properly ligature, kerned and so forth and you printed it in PDF, the PDF will be able to properly reproduce (faithfully reproduce) the text in any display for any computer or digital device.

Sure, PDF can be improved upon. Right now, it's what I stick to though because I find the other formats unsatisfying. And to my eye (even if not a professional typographer's or typesetter's eye) the other formats really is glaringly ugly in producing text.

My eyes bleed when I see that ;-)

---------------------------------------
Edit: by the way, I should say that I still find kerning very difficult to know how to do well. When I try to produce something aesthetically pleasing, I like most people rely on automatic kerning because I find it very difficult to make it nice looking. When I can see some areas where automatic kerning is not working properly I do try to adjust it manually but a lot of the times I can't get the spacing correctly. Knowing how to kern aesthetically is really difficult. It takes a lot of training and experience.

[thibaulthalpern]

Originally Posted by Sonist

I don't understand the point.
I can see the validity of kovidgoyal's "semantic" argument (although I don't see why it can't be largely addressed by future development of PDF,) but don't see the issue here.

I don't know too much about PDF, but I do know that you can tag texts in PDFs as paragraph, header, caption, font, etc. etc. But, someone who is much more knowledgeable about PDFs should actually chime in.

I don't produce PDFs professionally so I don't go into the intricacies of what it can and cannot do.

[sirbruce]

While that's an interesting point, what you describe should be *purely* a presentation-layer issue. The "f" and the "i" should be unchanged in the data stream. The reader will have to parse that and determine if it needs to do anything special fontwise or not. Relying on the composer of the data to "suggest" a different piece of data to be displayed at that location seems to me to be bad practice.

[Sonist]

Originally Posted by sirbruce

... Relying on the composer of the data to "suggest" a different piece of data to be displayed at that location seems to me to be bad practice.

But relying on a relatively primitive reader program is a good practice?! And better than relying on the composer of the data?!

I thing I am missing something here, or you did not read thibaulthalpern's post.

[sirbruce]

Originally Posted by Sonist

But relying on a relatively primitive reader program is a good practice?! And better than relying on the composer of the data?!

I thing I am missing something here, or you did not read thibaulthalpern's post.

From what I read he's saying, he's saying here, I've got some data with the letters "fi" next to each-other in it. Now, because *I* think you're a human reader, and *I* think your fonts probably can't handle that, *I* am going to *change* that data to some strange glyph. But it's okay, because I'm relying on you to have *my* special software to re-render that glyph back into something human readable. But if you don't, your font may come up with something totally not readable. In any case, the original data is *gone* or hidden. This makes the data inscrutable to many devices unless they special case for such a glyph.

Now, it perfectly makes sense for a specialized reader software, if it suspects a font problem, to make this substition at the display level. It makes very little sense for the *composer* software to do so, changing the data before it's written into a certain format. Even if the format didn't change the data but merely provided some suggestion of an alternative data set, I would be hesitant to say that's a good idea; not only does it cause data bloat, but could easily lead to more trouble trying to parse and display your idiosyncratic format.

[Jack Tingle]

Originally Posted by Sonist

I don't understand the point.

Take a non-image PDF and open it on your PC. Now, zoom in. The text stays nice and crisp.

Similarly, the Adobe Mobile SDK 9 reflows simple text, and allows enlarging the font. But it also can deal with complex, fixed layouts, where required.

So, what exactly is the problem?

Take a "typical" non-image PDF and open it on your "3.5 in. screen PDA or smartphone". Don't assume ebook customers are willing to shell out $300 for a single purpose device with questionable useful life and deliberately limited capabilities.

Assume most users don't have access to a SDK, nor do they care to. Note that most publishers wouldn't know how to set up a properly tagged, reflowable ebook if their lives depended on it, and most don't care, because they're paper oriented. Add in the fact that most folks really don't care about kerning, hyphenation, w&o, and ligatures, they just want to find out if Lord Hunkly manages to rip Goody Hottwif's bodice in chapter 5. Flush left/ragged right text is fine for them.

They don't really care where the page breaks fall, because they're reading text segmented by chapters anyway. They would like to be able to set serif vs. sans serif type and text size to fit their personal preference, screen characteristics, and/or visual capabilities. That's one reason they like ebooks better than paper ones.

With those givens, PDF is not a usable ebook format. In fact, it's a damned annoying format, because some ebooks people might really want to read are locked up in it and not easily transferrable to the portable devices that many people use for reading in the real world.

Regards,
Jack Tingle

[Xenophon]

Originally Posted by thibaulthalpern

This is an old, old problem. Anyone who is knowledgeable or a professional in graphic design and typography please chime in.[SNIP some discussion of ligatures]

With advanced ligature capabilities, the "fi" becomes a glyph (see image, top left)which I obviously cannot show you textually because this feature is not available in this forum nor is it available generally on the web. So, I demonstrate again with the graphic below. Notice how in proper ligature format, the "dot" of the "i" has disappeared. In fact, it no longer exists! The hook of the "f" takes over the space that originally would have been inhabited by the "dot" of the "i".

[SNIP more discussion of ligatures]

Now, this only touches the surface of typographical problems. There are many problems of which mentioned previously by someone else was kerning. Automatic kerning still isn't good enough to be used totally professionally. If you want a properly kerned text, you really still need to do it by eyesight.

And the problem goes on and on and on. If a piece of text was properly ligature, kerned and so forth and you printed it in PDF, the PDF will be able to properly reproduce (faithfully reproduce) the text in any display for any computer or digital device.

Sure, PDF can be improved upon. Right now, it's what I stick to though because I find the other formats unsatisfying. And to my eye (even if not a professional typographer's or typesetter's eye) the other formats really is glaringly ugly in producing text.

My eyes bleed when I see that ;-)

---------------------------------------
Edit: by the way, I should say that I still find kerning very difficult to know how to do well. When I try to produce something aesthetically pleasing, I like most people rely on automatic kerning because I find it very difficult to make it nice looking. When I can see some areas where automatic kerning is not working properly I do try to adjust it manually but a lot of the times I can't get the spacing correctly. Knowing how to kern aesthetically is really difficult. It takes a lot of training and experience.

Originally Posted by sirbruce

While that's an interesting point, what you describe should be *purely* a presentation-layer issue. The "f" and the "i" should be unchanged in the data stream. The reader will have to parse that and determine if it needs to do anything special fontwise or not. Relying on the composer of the data to "suggest" a different piece of data to be displayed at that location seems to me to be bad practice.

Originally Posted by sirbruce

From what I read he's saying, he's saying here, I've got some data with the letters "fi" next to each-other in it. Now, because *I* think you're a human reader, and *I* think your fonts probably can't handle that, *I* am going to *change* that data to some strange glyph. But it's okay, because I'm relying on you to have *my* special software to re-render that glyph back into something human readable. But if you don't, your font may come up with something totally not readable. In any case, the original data is *gone* or hidden. This makes the data inscrutable to many devices unless they special case for such a glyph.

Now, it perfectly makes sense for a specialized reader software, if it suspects a font problem, to make this substition at the display level. It makes very little sense for the *composer* software to do so, changing the data before it's written into a certain format. Even if the format didn't change the data but merely provided some suggestion of an alternative data set, I would be hesitant to say that's a good idea; not only does it cause data bloat, but could easily lead to more trouble trying to parse and display your idiosyncratic format.

thibaulthalpern is quite correct in his discussion of typographical issues. And the later posters I've quoted are equally correct that this should be a presentation-layer issue rather than a change that is made in the underlying document format.

And it IS a presentation-layer issue in any piece of modern Mac software that didn't come from Microsoft. The standard OS text utilities include so-called "advanced typography support," which includes automatic ligatures and some auto-kerning support. The ligature support is really slick and entirely transparent to the underlying application code. The kerning support is also transparent for the underlying application, but is not (yet) anywhere near as good as carefully performed hand-optimized kerning done by an expert. But it is better than my layperson's novice-quality efforts.

It seems to me that you folks are having an unnecessary argument. The problem is not that PDF is useless, nor is it that ePub (for example) can't ever do a good enough job of presentation.

Rather, the problem is that PDF is really centered around a fix-size page. And that makes it significantly less usefull when the screen of your viewing device is significantly different from the intended page size. Similarly, the issues with ePub come (mostly) from limitations in the display software, not from limitations in the format itself.

So PDF loses when I have a 6"-diagonal screen and your beautifully-layed-out document is optimized for a 13.9" screen! But ePub (today) doesn't even come close to the level of the best hand-optimized layouts for any particular size. And, in some abstract sense, what we really want is the best of both!

But with today's technology an ePub is usable on many screen sizes, where a PDF usually isn't.

Xenophon

[thibaulthalpern]

I'm not talking about a change in underlying document format. I'm talking about why at this point PDF still trumps ePub format when talking about faithfulness to design.

It shouldn't be up to the software display program to determine whether the "fi" should be ligatured. This should be a decision made by the human who creates the document. There are instances when "fi" should NOT be ligatured because the font is inappropriate to do so, or the presentation of the text does not call for ligatured glyphs. In certain scientific technical documents, I don't see ligatures being useful.

If we are depending on reader software to determine whether ligatures should or should not happen, it's like the problem with Microsoft Word's autoformat which is unfortunately mostly a hinderance because it cannot interpret the context in which we are working in and thus does not know whether we want something formatted in x-style or y-style.

The reason PDF trumps other digital formats that I know of is precisely because of its superiority in display presentation. That's basically it for me. The PDF format doesn't determine whether some is kerned so or ligatured so, because it's all set up by the human being (or the human-software interaction) PRIOR to the creation of the PDF.

PDF format continues to change. There are different versions of PDF (see: http://en.wikipedia.org/wiki/Pdf#Versions) Furthermore, PDF can reflow but this depends on if the person creating the PDF is savvy in using the accessibility features. Please see http://en.wikipedia.org/wiki/Pdf#Accessibility.

Or better yet, let me quote at length form Wikipedia:

Moreover, tagged PDFs can be re-flowed and magnified for readers with visual impairments [emphasis mine]. Problems remain with adding tags to older PDFs and those that are generated from scanned documents. In these cases, accessibility tags and re-flowing are unavailable, and must be created either manually or with OCR techniques. These processes are inaccessible to some disabled people. PDF/UA, the PDF/Universal Accessibility Committee, an activity of AIIM, is working on a specification for PDF accessibility based on the PDF 1.6 specification.

One of the major problems with PDF accessibility is that PDF documents have three distinct views, which, depending on the document's creation, can be inconsistent with each other. The three views are (i) the physical view, (ii) the tags view, and (iii) the content view. The physical view is displayed and printed (what most people consider a PDF document). The tags view is what screen readers read (useful for people with poor eyesight). The content view is displayed when the document is re-flowed to Acrobat (useful for people with mobility disability). For a PDF document to be accessible, the three views must be consistent with each other.

Source: http://en.wikipedia.org/wiki/Pdf#Accessibility

[DaleDe]

Originally Posted by sirbruce

While that's an interesting point, what you describe should be *purely* a presentation-layer issue. The "f" and the "i" should be unchanged in the data stream. The reader will have to parse that and determine if it needs to do anything special fontwise or not. Relying on the composer of the data to "suggest" a different piece of data to be displayed at that location seems to me to be bad practice.

I agree with you but unfortunately it is current practice for PDF. I have tried to read several PDF's on portable devices only to find missing characters in the text due to this use of fi vs. fi (& #64257; ) ligature. It is a pain.

Dale

[taosaur]

Originally Posted by Xenophon

So PDF loses when I have a 6"-diagonal screen and your beautifully-layed-out document is optimized for a 13.9" screen! But ePub (today) doesn't even come close to the level of the best hand-optimized layouts for any particular size. And, in some abstract sense, what we really want is the best of both!

But with today's technology an ePub is usable on many screen sizes, where a PDF usually isn't.

What we will likely end up with for the foreseeable future is not the best of both, but both. For some texts, like science textbooks and magazines, trying to separate content from design for display on a 6" screen is futile. Pulp novels and academic journals, on the other hand, are probably more readable than the original on even a small text-optimized display.

If you think about it, 'books' is a very diverse class of objects. We can say that they're all made of paper, but in reality the qualities of that paper and the means of embedding information thereupon approach infinity. What I think we're seeing is a sharper delineation between single-stream texts, which can move entirely into the digital realm once the price of admission falls, and books that serve as tools and/or objets d'art. Trying to cram the latter categories into an eInk device because it worked for detective novels and "they're all books" may be premature. We may find that the best digital translations for many texts aren't something we'd consider an 'ebook' at all.

So maybe PDF is not an ebook format, because 'ebook' is a much narrower category than 'book.'

[Xenophon]

Originally Posted by thibaulthalpern

I'm not talking about a change in underlying document format. I'm talking about why at this point PDF still trumps ePub format when talking about faithfulness to design.

Of course PDF provides better fidelity to design -- that has always been it's prime purpose! All the accessibility and reflow features arrived later as afterthoughts.

Originally Posted by thibaulthalpern

It shouldn't be up to the software display program to determine whether the "fi" should be ligatured. This should be a decision made by the human who creates the document. There are instances when "fi" should NOT be ligatured because the font is inappropriate to do so, or the presentation of the text does not call for ligatured glyphs. In certain scientific technical documents, I don't see ligatures being useful.

Well, the standard Mac software provides the necessary interfaces to let the user control both default choices about ligatures and also any specific instance of the appearance (or non-appearance) of a ligature. My main point was that the default settings generally do quite nicely. It does seem to me, however, that a properly designed font should include the ligatures that are appropriate for that font, and should omit any ligatures that are inappropriate. And the advanced typography support doesn't conjure up ligatures out of nowhere -- it selects only from among the ligatures that are present in the font being used.

As for scientific technical documents, I prefer to have (standard) ligatures in running text -- they make it easier to read. I suppose that some of the most decorative and extreme ligatures might not belong there. On the other hand, ligatures clearly do not belong in formulae or source code. But I can trivially make this choice via the standard display settings (for default, styles, etc.).

Originally Posted by thibaulthalpern

If we are depending on reader software to determine whether ligatures should or should not happen, it's like the problem with Microsoft Word's autoformat which is unfortunately mostly a hinderance because it cannot interpret the context in which we are working in and thus does not know whether we want something formatted in x-style or y-style.

The current state of the art in software certainly fails to make the correct choices without some user input. But for most fiction (for example) a single default setting handles 99.9% of the book -- that is, all the running text. And even titles probably look better with the standard simple ligatures. Further, the state of the art in software continues to improve over time.

Originally Posted by thibaulthalpern

The reason PDF trumps other digital formats that I know of is precisely because of its superiority in display presentation. That's basically it for me. The PDF format doesn't determine whether some is kerned so or ligatured so, because it's all set up by the human being (or the human-software interaction) PRIOR to the creation of the PDF.
[SNIP a bunch of good text about the PDF format]

That's all true -- as long as the display used for the presentation is the same as the designer's intended display.

But while ligature choices likely should survive size-changes and reflow, I'm far less certain about kerning. And the vast bulk of the rest of the careful human design for layout is usually, well, wasted when my viewing display differs substantially from your intended layout in terms of size, aspect ratio, DPI, contrast, color (or lack thereof), etc.

For example, there are plenty of fonts that are absolutely gorgeous when printed at 1200DPI but are unreadable on my ~180DPI Sony PRS-700 -- even when the physical measurements of the characters are identical! And your beautifully hand-tuned layout for an 8.5"x11" page (or A4 page!) won't work well on my 6"-diagonal reader.

On the other hand, the current presentation of auto-reflowed formats such as ePub is often significantly lacking as well. We have poor support for full justification (with embarrassingly hideous rivers of white-space), lousy or just plain missing auto-hyphenation, no ligatures ever, and on and on.

The point I've been trying to make here (at far too much length) is that when reflow is required PDF is even worse than formats like lrf or ePub. Meanwhile the reflow-able formats don't display as nicely as a well-designed hand-optimized document (which must be presented via PDF, or Postscript, or an image file of some sort or...).

The different kinds of formats have different strengths and weaknesses. As auto-layout and presentation software gets better, the reflow-able formats will look better and better. As auto-reflow from PDF gets better, that nice hand-optimized layout will degrade less and less for other page sizes. But today, PDF is the right choice for fixed size and nearly hopeless for reflow. And lrf/epub/etc. are decent for reflow, and ugly for fixed-size presentation.

You pays your money and makes your choice. One format does not yet fit all!

Xenophon

[thibaulthalpern]

Originally Posted by Xenophon

The different kinds of formats have different strengths and weaknesses. As auto-layout and presentation software gets better, the reflow-able formats will look better and better. As auto-reflow from PDF gets better, that nice hand-optimized layout will degrade less and less for other page sizes. But today, PDF is the right choice for fixed size and nearly hopeless for reflow. And lrf/epub/etc. are decent for reflow, and ugly for fixed-size presentation.

I'm responding to the portion I bolded and italicised. My response:

Really? It's horrible for reflow? That depends on how the PDF document was created. I have some PDFs that reflow just fine. These are the "Very Short Introduction" series published by Oxford University Press.

If you create a PDF simply from "print to PDF" types of command, it's likely that the reflow will be horrible.