support for XXspace chars in epub

[AlanHK]

I thought I was clever when I worked out how to add pseudo-kerning between a single and double quotemarks, so they didn't turn into a triple.

I noticed some books put a normal space, [“ ‘]
but that allows a linebreak between the quotes. I thought   looked too wide, so I use : [“ ‘].

And later when indenting some poetry, I used   (em-space) characters.

That looked okay on Sigil. And I'm told it works on iPad iBooks.

But I noticed both these characters display as black boxes in Calibre's ebook-reader.

Is that just a bug in Calibre, or are these characters not officially meant to be used in ePubs?

[Jellby]

Using named entities may be problematic in epubs, but the black boxes are probably due to the font lacking those characters, or to a buggy renderer, every Unicode character is allowed in epubs.

[DaleDe]

Originally Posted by Jellby

Using named entities may be problematic in epubs, but the black boxes are probably due to the font lacking those characters, or to a buggy renderer, every Unicode character is allowed in epubs.

Exactly, a box is a clear indication of a missing font. All of unicode is permitted but few eBook readers (if any) have all the font glyphs in their repertoire. You can embed a font set to fix this.

Dale

[AlanHK]

Originally Posted by Jellby

Using named entities may be problematic in epubs, but the black boxes are probably due to the font lacking those characters, or to a buggy renderer, every Unicode character is allowed in epubs.

Sigil converted the emspace name to Unicode, but it still displayed as a box.

I have Georgia as the default serif font for Calibre, and also in Sigil, but it displays correctly in Sigil but not Calibre reader.
It also displays correctly in Calibre editor preview.

Georgia is a pretty complete font, and I tried Times New Roman and that's also boxes.

And I converted the epub to mobi using Kindle previewer. The spaces display in everything, in Georgia.

So I think the font and code is valid, it's just Calibre.

[rubeus]

  is not supported by my PRS 650 even if i have fonts in the ebook. Sigil is not trust worthy here, cause it uses QT5 which will render and allow much more as declared in the epub Standard, e.g. Sigil will follow all pseudo-classes, but no epub2 reader will follow this style.

[Notjohn]

Speaking of em spaces, I am reading The Far Side of Paradise, Mizener's wonderful biography of F. Scott Fitzgerald. It was published in the 1950s, I think. I was astonished to see a double-wide space following every sentence, much as we were taught in typing class at Brewster Free Academy (using Army-surplus Remington manual typewriters). I have never before seen such a thing in a published book. What was Houghton Mifflin thinking?

I apologize for hijacking the thread, but to me this is a momentous discovery (despite the fact that I'd read the book years ago) and I just had to tell someone!

[AlanHK]

Originally Posted by rubeus

  is not supported by my PRS 650 even if i have fonts in the ebook. Sigil is not trust worthy here, cause it uses QT5 which will render and allow much more as declared in the epub Standard, e.g. Sigil will follow all pseudo-classes, but no epub2 reader will follow this style.

Interesting.
I'm using XP, which doesn't support QT5 apps.
So I'm using an older version of Calibre and its minions, which shows the problem.


I tried all the spaces in Sigil's "insert" :
nbsp
ensp
emsp
thinsp
shy
nnbsp

of these only nbsp and shy worked, the rest are black boxes.

Great.
You don't even need a glyph to display a space. Any font should work.
I expect issues with Chinese or Hieroglyphics, but spaces should be trivial and work in everything.

So now I have to use stupid thick nbsp instead of thin.
And I replaced each em with 3 nbsp, which looks really stupid in code but works.

Originally Posted by Notjohn

Speaking of em spaces, I am reading The Far Side of Paradise, Mizener's wonderful biography of F. Scott Fitzgerald. It was published in the 1950s, I think. I was astonished to see a double-wide space following every sentence

Books printed early last century had different conventions for spacing. Wide space after sentences, and more before colons, e.g. And they used a lot more capitals.

Look at a few random old books on Google Books or at your local library.
By the 1950s it would be uncommon but obviously still acceptable.

But now of course with easy DTP and all the self published crap, double spacing, straight quotemarks and every other typographic horror is to be seen.

[eschwartz]

Actually, you do need a glyph for spaces. Although I believe they can all reference the same blank glyph?

If the space is not recorded in the font, how does the software know it's a space?

[AlanHK]

Originally Posted by eschwartz

Actually, you do need a glyph for spaces. Although I believe they can all reference the same blank glyph?

I meant, it doesn't need a shape, not one pixel or line. All it needs to print is the width. So I would hope that any Unicode font at least would have all these space "glyphs" by default, as they don't require any design, just one width which is probably just the normal space width multiplied by a standard factor.

But I guess my hope is in vain.

Originally Posted by eschwartz

If the space is not recorded in the font, how does the software know it's a space?

The software should know it's a space without reference to the font.

If you edit a font and put a glyph in a space slot, the result can be poor. The software will break lines at this character, for instance, expecting it to be a space between words.


I use an old DTP app that writes to postscript, which I have studied as I occasionally modify it to get some effect.
When doing kerned and justified text it outputs one character, then a number to advance before the next. So any spaces are elastic.

I know epub layout doesn't give us that control, but it demonstrates one simple way arbitrary spaces can be created whether they're in the font file or not.

[pdurrant]

Originally Posted by AlanHK

I thought I was clever when I worked out how to add pseudo-kerning between a single and double quotemarks, so they didn't turn into a triple.

Rather than adding a character, wrap one of the punctuation marks in a styled <span>, and add some space with an margin setting in the CSS. You can then make the gap exactly how wide you want.

[eschwartz]

Originally Posted by AlanHK

The software should know it's a space without reference to the font.

If you edit a font and put a glyph in a space slot, the result can be poor. The software will break lines at this character, for instance, expecting it to be a space between words.


I use an old DTP app that writes to postscript, which I have studied as I occasionally modify it to get some effect.
When doing kerned and justified text it outputs one character, then a number to advance before the next. So any spaces are elastic.

I know epub layout doesn't give us that control, but it demonstrates one simple way arbitrary spaces can be created whether they're in the font file or not.

That is all fine and well, but your coulda woulda shouldas aren't the way fonts work.

For example, it is entirely valid and within the defined purpose of font files to redefine a space as some arbitrary glyph. I can guarantee you there is someone, somewhere, who has good reasons to do just that... no matter how bad you may feel it looks.

There is no reason why rendering software should make a special exception just for *spaces to hardcode their definition rather than allow the font to define it. In fact, there is every reason NOT to.

[Jellby]

For spacing between quotes, I prefer to rely on font kerning and, since most fonts don't have proper kerning for these pairs, I modify the font I use in my reader.

[AlanHK]

Originally Posted by eschwartz

That is all fine and well, but your coulda woulda shouldas aren't the way fonts work.

For example, it is entirely valid and within the defined purpose of font files to redefine a space as some arbitrary glyph. I can guarantee you there is someone, somewhere, who has good reasons to do just that... no matter how bad you may feel it looks.

There is no reason why rendering software should make a special exception just for *spaces to hardcode their definition rather than allow the font to define it. In fact, there is every reason NOT to.

I know how fonts work. I've edited and customised a few with font editors over the years.
So I know you can put any glyph in any slot. But if you do it with a text font and not just a grabbag of symbols, it will screw up on the page.

There are excellent reasons for software to treat space characters specially, and they do, as I gave an example earlier. How else can it know where to wrap a string of text?
Layout software will break at a "space" regardless if you put a actual glyph there.

Anyway, all these "special" spaces could be created on the screen/page regardless if they have a slot in the font file. DTP page layout software does that. But ebook rendering is less predictable.

[AlanHK]

Originally Posted by Jellby

For spacing between quotes, I prefer to rely on font kerning and, since most fonts don't have proper kerning for these pairs, I modify the font I use in my reader.

That's how I do it for print layout. I edited a bunch of kerns (mostly for punctuation) into my standard text fonts long ago.

But what epub readers have kerning? I think that's even less likely to be supported than these "exotic" spaces.

Originally Posted by pdurrant

Rather than adding a character, wrap one of the punctuation marks in a styled <span>, and add some space with an margin setting in the CSS. You can then make the gap exactly how wide you want.

Thanks, I guess that will work.
Maybe I could use "letter-spacing".
I'll play with it.

[Jellby]

Originally Posted by AlanHK

But what epub readers have kerning? I think that's even less likely to be supported than these "exotic" spaces.

Both my readers (Kobo Aura and Cybook Orizon) support kerning, even my old Gen3 supported kerning if I remember correctly. They support using custom fonts too, without embedding, so I can enjoy kerning (and ligatures) in any book.