Why is it so hard to preserve blank lines?

[JSWolf]

Originally Posted by Robotech_Master

Well, how about that? Scrivener's developer has chimed in with a more complete explanation of why Scrivener uses non-breaking space paragraphs. It effectively boils down to Scrivener using Apple's HTML 4.0 conversion tool to turn its documents into EPUB-ready HTML, and that's how Apple does it.

I've seen output from how Pages makes ePub. It's ghastly. Don't follow Apple.

[dgatwood]

Originally Posted by JSWolf

I've seen output from how Pages makes ePub. It's ghastly. Don't follow Apple.

To be fair, they've gotten better. When I started writing my first three novels, I was using AppleWorks. At some point, I realized that I wanted more advanced formatting than was feasible using that tool, so I created a bunch of tools that would let me format things in various ways—one formatting style for drafts (double-spaced, squiggle underscore for bold, etc.), another style for final content in hardcover page size, another style for paperback page size, etc.

But in order to do that, I had to find a way to get content out of AppleWorks. So obviously I used its HTML export. Here's a snippet of actual output (edited only to wrap the lines and replace carriage returns with usable line breaks):

<P ALIGN=CENTER><I><BLOCKQUOTE>Chief engineer’s log for April 6, 2333<BR>
</P>
<P><BR>
It has been three days since I upgraded the OS on the core machines to
the latest version.  Critical systems are still being handled by my machine
in engineering, while non-critical systems have been transitioned gradually
back to the primary cores.<BR>
We have recently begun to experience new failures similar to those encountered
previously.  Trivial investigation suggests that, while many of the attacks may
have occurred through known mechanisms, other forces may be at work.</I><BR>
</BLOCKQUOTE><BR>
</P>

IIRC, content that looked like this was actually compatible with Netscape Navigator 1.0, and after it broke in Netscape Navigator 2.0, Apple never bothered to update their HTML exporter to make the tags match.

Note that the software in question wasn't discontinued until 2007....

[JSWolf]

Originally Posted by dgatwood

To be fair, they've gotten better. When I started writing my first three novels, I was using AppleWorks. At some point, I realized that I wanted more advanced formatting than was feasible using that tool, so I created a bunch of tools that would let me format things in various ways—one formatting style for drafts (double-spaced, squiggle underscore for bold, etc.), another style for final content in hardcover page size, another style for paperback page size, etc.

But in order to do that, I had to find a way to get content out of AppleWorks. So obviously I used its HTML export. Here's a snippet of actual output (edited only to wrap the lines and replace carriage returns with usable line breaks):

<P ALIGN=CENTER><I><BLOCKQUOTE>Chief engineer’s log for April 6, 2333<BR>
</P>
<P><BR>
It has been three days since I upgraded the OS on the core machines to
the latest version.  Critical systems are still being handled by my machine
in engineering, while non-critical systems have been transitioned gradually
back to the primary cores.<BR>
We have recently begun to experience new failures similar to those encountered
previously.  Trivial investigation suggests that, while many of the attacks may
have occurred through known mechanisms, other forces may be at work.</I><BR>
</BLOCKQUOTE><BR>
</P>

IIRC, content that looked like this was actually compatible with Netscape Navigator 1.0, and after it broke in Netscape Navigator 2.0, Apple never bothered to update their HTML exporter to make the tags match.

Note that the software in question wasn't discontinued until 2007....

That is some poor code. You do not need a <BR> (should be <BR/>) in the places it is in. It's poor code and gives you no control. Having a full line between paragraphs is very ugly. By default, a <blockquote> already have top and bottom margin spacing. The <BR> is adding even more space. Awful code that you really should clean up. You can change how the spacing between paragraphs by adjusting the margins for <p>. Like I said, Apple makes poor HTML code that you may need to clean up by hand if S&R is not going to help.

[dgatwood]

Originally Posted by JSWolf

That is some poor code. You do not need a <BR> (should be <BR/>) in the places it is in. It's poor code and gives you no control. Having a full line between paragraphs is very ugly. By default, a <blockquote> already have top and bottom margin spacing. The <BR> is adding even more space. Awful code that you really should clean up. You can change how the spacing between paragraphs by adjusting the margins for <p>. Like I said, Apple makes poor HTML code that you may need to clean up by hand if S&R is not going to help.

You missed the big problem, which is:

<p><i><blockquote>...</p>
<p>...</i></blockquote></p>

The italic starts outside the blockquote tag and ends inside it. And the blockquote starts inside one paragraph and ends inside the next one, despite the fact that it is illegal to have a blockquote inside a paragraph tag at all, much less a single blockquote tag that wraps parts of the interior of two paragraphs.

The content in question doesn't even parse, much less render sanely.

I have a script whose sole purpose is to fix the tag matching problems so that I can feed it into another script that turns that mess into sane DocBook XML, which I then emit as proper XHTML. My publishing toolchain is utterly insane, and a decent chunk of that insanity is Apple's fault. (The rest is mostly LaTeX's fault.)

[Robotech_Master]

So, I'm curious. Since I started this thread, and wrote these articles—

http://www.teleread.com/how-epubs-re...eak-standards/

http://www.teleread.com/blank-line-i...pub-standards/

—has anything changed?

Is there a better way to indicate section breaks that is honored by the majority of e-readers yet?

[DaleDe]

The best way is usually to use CSS to make the separation. It is smart enough to not have a space line as the top line in a new page. Honoring a CR or CR/LF causing wrapping problems as well as spacing problems. In the interest of easy reading an indented first line in a paragraph (done with CSS) is a better option than a blank line to separate paragraphs. It places more text on a page so less page turns. A blank line is good for additional separation and the nbsp paragraph is fine for this. Readers should support both but CSS is always best if it suits. ePub 3 depends even more on CSS that 2 does.

Dale

[JSWolf]

Originally Posted by Robotech_Master

So, I'm curious. Since I started this thread, and wrote these articles—

http://www.teleread.com/how-epubs-re...eak-standards/

http://www.teleread.com/blank-line-i...pub-standards/

—has anything changed?

Is there a better way to indicate section breaks that is honored by the majority of e-readers yet?

Yes there is...

body {
  widows: 1;
  orphans: 1;
  margin-top: 0;
  margin-right: 0;
  margin-bottom: 0;
  margin-left: 0;
  text-align: justify;
}
p {
  margin-top: 0;
  margin-bottom: 0;
  text-indent: 1.2em;
}
.spacebreak {
  padding-top: 2em;
  text-indent: 0;
}

<p>Some random line.,/p>
<p class="spacebreak">This is another section.</p>

Because you are using windows and orphans of 1, you don't have extra space at the bottom. And that means the padding of 2em shows up instead of getting lost with widows and orphans. The padding is used instead of margin because padding works better for a section break when it's at the bottom or top of the screen.

[Tex2002ans]

Originally Posted by Robotech_Master

So, I'm curious. Since I started this thread, and wrote these articles—
[...]
—has anything changed?

What's the purpose of these blank lines?

If they're intended for spacing, then I agree with JSWolf, use CSS with padding-top:

Spoiler:

.spacebreak {
  padding-top: 2em;
  text-indent: 0;
}

<p>Some random line.</p>

<p class="spacebreak">This has a blank gap above.</p>

padding-top doesn't get gobbled up if it occurs at the top of a screen, where margin-top typically does.

Originally Posted by Robotech_Master

Is there a better way to indicate section breaks that is honored by the majority of e-readers yet?

If the blanks are intended as actual scenebreaks, then I would again stress murg's post #12 where he mentioned centered asterisks.

Spoiler:

HTML:

<p class="scenebreak">* * *</p>

CSS:

p.scenebreak {
    text-align: center;
    margin-top: 1em;
    margin-bottom: 1em;
}

Using symbols for a scenebreak ensures maximum compatibility, and the advantages are numerous:

• Works with/without CSS.
• Works in any format.
  
  • Even survives conversion from EPUB->TXT.
• Works with any font.
• Can know a new section has started, even if it lands at the very bottom/top of screen.
• Accessibility.

I've discussed scenebreaks (and why you should use asterisks) in these Reddit posts:

"How to Format Scene Breaks?"
"Any tips on adding ornamental art in Sigil or Scrivener?"

I'll reproduce a lot of the reasoning here:

Works With/Without CSS

A blank gap + no-indent based purely on CSS fails in readers that:

• Override Publisher settings
  
  • Moon+ Reader is a famous one.
• Don't support CSS.

Asterisks can survive and make sense with/without CSS:

Test1 (Asterisks)



Test2 (No Asterisks)



You can see Test2Without (the example in the bottom right). The new scene looks no different than any other paragraph.

Also see below (many alternate formats don't use CSS).

Works In Any Format

Remember, ebooks can be converted into many different formats (NOT just EPUB/MOBI).

Asterisks can survive EPUB->TXT conversion + can even survive "non-standard" formats such as forum posts. This allows you to easily copy/paste a chapter from your book onto MobileRead for example.

Works With Any Font

Why use asterisks over other many other symbols? I'll quote one of my Reddit posts above:

The reasons why asterisks work best is it's basic ASCII... so pretty much any device and any font anywhere would have it included.

All 136/136 fonts on my computer have the asterisk *.

When you get to more obscure symbols, it's more likely those fonts may be missing it:

For example, see "Fleuron" on Wikipedia:

• ❧ Rotated Floral Heart Bullet (U+2767)

20/136 of the fonts installed on my computer have that.

Ereaders have even less fonts to choose from, and usually aren't the best at "falling back" to another font (like Firefox/Chrome+Word/LibreOffice might).

Asterisks don't rely on embedded fonts, and would work with whatever the font the user chooses.

Top/Bottom of Screen

I somewhat touched on this with the padding-top/margin-top discussion.

Also see this article on EPUB Secrets: "User Experience: What Works, and How?", heading "Section Breaks":

Section Breaks

Or narrative pauses or time breaks or whatever you choose to call them: Simon Collinson brought this up:

I’ve always tried to convince editors and clients to use some kind of marker to indicate a pause. It’s common in print to add a few small bullets or an ornament when a break falls at the bottom of a page, just so the reader knows to breathe before entering a new timeframe at the top of the next page (a flush-left paragraph may not be enough to clue them in).

In an ebook that pause is very likely to fall at the bottom of a screen; we have no way of knowing how our readers are setting their preferences. So, to me, it’s a no-brainer to add something as an aid.

This doesn't just effect "screens" in an ebook, it also effects all paged formats as well (even Print books).

If typographers use a blank gap, they must be very careful that new scenes don't fall at the very bottom/top of a page.

Print has full control over the final layout... ebooks... no.

Fleurons handle that problem completely.

Accessibility

Blank paragraphs or "visual only" clues are absolutely awful for Accessibility or Text-to-Speech (TTS) reasons as well.

TTS will continue reading the next paragraph no different from the previous one. No pause, no nothing.

Have you ever read a book in Moon+ Reader with TTS that relied on blank gaps? I have... and the characters teleport around. One paragraph they're eating a chicken sandwich in their kitchen, and next you're on a spaceship flying towards the moon.

[Hitch]

Agreed with Tex. The easiest way--which is the gist of your archived posts on Teleread, as opposed to the right way, taken all in--is to use a symbol or fleuron or asterisms. That will work with all systems--I've yet to find one where it didn't--and avoid the issues of the nine-bajillion free "eReading apps" that all recognize whatever they want, and ignore pesky things like the IDPF Standards. (And yeah, Apple, I'm lookin' at you, too!).

{shrug}. Plus to speak to your other point, this way authors don't have to learn to code their books "properly." They can hit enter, center, type 3 asterisks, hit enter and bob's-yer-uncle.

Personally, I prefer top-margins created with CSS, but it's true that in the less-well-known readers and many reading apps, this method can be/might be/could be ignored. (This is why we tell our clients we'll warranty our work on the major devices, but not software eReaders--too many variables, too many obscure pieces, etc. It'll be a cold day in hell before I contort myself or my company and bookmakers to make a book work on Obscure eBook Reader #1,911, you know, just because someone found it on their apps store and it was FREE. I can't tell you how many times we get complaints about a book that looks fantastic on the biggies, but doesn't display EXACTLY the way that the author wanted on some dingbat reading app. And of course, it's never the simpler books that have this issue; it's always the publishers that have had us jump through all sorts of fiddly hoops, that seem to find the least-used eReading apps in existence...)

Even using Scrivener, that should be fairly simple, I'd think?

Hitch

[theducks]

Tex
If you use a Image break, it should have an alt="meanwhile, later, scene break...." <select for the least jaring

The reader is supposed to say the Alt phrase

[Hitch]

Originally Posted by theducks

Tex
If you use a Image break, it should have an alt="meanwhile, later, scene break...." <select for the least jaring

The reader is supposed to say the Alt phrase

Ducky:

Thanks for reminding us. :-)

Hitch

[jackie_w]

Originally Posted by theducks

The reader is supposed to say the Alt phrase

"Supposed" being the operative word I haven't yet seen a reading app with TTS which uses the <img> alt text - and I have tried a lot of them.

Even if/when app developers do get around to implementing this feature my guess is that users will immediately ask for a "disable" option because, if the ebook creators have set non-blank alt text, it's text to help themselves, not the user, e.g. a page number or the image's filename. I have a vague memory that some Brandon Sanderson books did try to use alt wisely.

As an aside, if everything worked as it should, I wonder what a good "TTS scenebreak phrase" might be. Anything longer than a couple of words might get very irritating if it occurs often. Personally I think some kind of sound might be better, but we're a long way from that in your average reading app

[Tex2002ans]

Originally Posted by theducks

Tex
If you use a Image break, it should have an alt="meanwhile, later, scene break...." <select for the least jaring

The reader is supposed to say the Alt phrase

Originally Posted by jackie_w

"Supposed" being the operative word I haven't yet seen a reading app with TTS which uses the <img> alt text - and I have tried a lot of them.

Exactly. Key word is "supposed" (I haven't run across one yet that reads alt either).

But remember, these ebooks won't necessarily could be read in ereaders only... they could be read using full-blown Screen Readers (and those DO support reading alt text, even if most ereader apps don't).

Side Note: We discussed this more in-depth in the Accessibility question: TTS and ligatures topic. This also discussed dropcap images with alt text.

* * *

But IF you're using images, definitely use meaningful alt text:

WebAIM article on Alternative Text

Even if/when app developers do get around to implementing this feature my guess is that users will immediately ask for a "disable" option because, if the ebook creators have set non-blank alt text, it's text to help themselves, not the user, e.g. a page number or the image's filename. I have a vague memory that some Brandon Sanderson books did try to use alt wisely.

No. Alt text SHOULD be meaningful (see WebAIM article above).

But yes, many programs output even worse than useless garbage like filenames. InDesign is notorious for this crap (but they're getting better. I think latest versions now let you more easily see alt tags.).

It would be better to have a blank alt="" than to have absolute gibberish in there.

Note: A similar issue with <title>. See my discussion on the topic (and more Accessibility links) here.

Note #2: For help in catching/generating/fixing some of this stuff... see Accessibility plugins like Access-Aide Plugin for Sigil.

Note #3: Also see this speech at ebookcraft 2017: In the Trenches with Accessible EPUB (Slides here.) This was a technical talk behind one of the checking tools, and discusses many of the things automated tools can't handle, but should still be kept in mind when designing/checking ebooks.

Originally Posted by jackie_w

As an aside, if everything worked as it should, I wonder what a good "TTS scenebreak phrase" might be. Anything longer than a couple of words might get very irritating if it occurs often. Personally I think some kind of sound might be better, but we're a long way from that in your average reading app

"New Scene"
"Scene Break"
"Subchapter"

I personally don't mind the TTS saying:

"asterisk asterisk asterisk" OR "... (Pause)" I currently get in Gitden Reader.

What would a real person do while reading? Take a long breath, take a longer pause, and continue reading the next scene.

Note: In the future, reading apps may also take more advantage of HTML5's <section>. Each scene/subchapter could be treated slightly differently.

* * *

And back to "Why Not Use Images for ChapterTitles/Scenebreaks"...

I already answered in the Reddit posts too (because one of the users was using images for his chapter titles), but I'll reproduce some of my reasoning here. :P

Here's the code example the user gave:

In the book I formatted with images for the headings, I used this code:

<h1 title="1"><img alt="1" src="../Images/01.gif" /></h1>

Then it shows up nicely on an auto-generated table of contents, and the alt gives a fallback for text-to-speech.

Why Not Use Images?

The disadvantages are numerous:

Images typically:

• Don't scale
• Scale awfully (too low resolution)
• Don't match user font color (Night Mode)

You also have to recall that users can:

• Set font sizes HUGE (so text will be huge, and then you have a microscopic little fleuron)
• Read on very tiny devices (think cell phone)
• Flip landscape/portrait (many people forget to test code in landscape and it breaks)

Ebooks that have horribly low resolution images around their headings or scenebreaks. They may have "looked nice" when the book first came out, but as devices became higher resolution, or you mess with settings, they look awful.

Images With Text Have Accessibility Issues

For example, readers may be using:

• OpenDyslexia font for reading disability
• Older folks may need HUGEEEE fonts
• Certain colors for Background+Text due to contrast issues

Even "normal readers" may be using:

• Press/Hold to access Dictionary
• Auto-Translate (think Google Translate English->French)
• Highlighting/Copying/Pasting Text
• Searching

These also don't work on "text in images".

Text in Images Doesn't Follow CSS

Style the chapter title with CSS if you want (large, bold, embed fancier font for headings, fancier CSS3 boxes/shadows, etc.), but the worst choice you can make is embed the entire chapter name/number into an image.

Similar to embedding captions within an image... you're going to be shooting yourself in the foot when later on you decide to now make captions italic+right-aligned instead of bold+centered.

[JSWolf]

The way I see it, if an eBook is made properly (no paragraph spaces & proper indents), then a section break of 2em of just space works because you get the space and then the next paragraph has no indent. So you can see the space followed by no indent.

What I've seen used that works is an <hr> of some width. 45% works pretty well.