Questions about converting ellipsis

[Bobosmite]

In my Word documents, I expand the spacing in the ellipsis by 3pt., but conversion ignores it and give me dot-dot-dot squeezed together. Typically, when it converts, it looks like this: TEXT ... TEXT or: TEXT...TEXT
By adding a space between the dots, this is how I want it to look: TEXT. . .TEXT

When converting to an ebook, can Calibre detect this as an ellipsis and allow the line to break before or after, or is that determined by the reading device? Also, does Calibre recognize non-breaking spaces?

[jackie_w]

Someone should correct me if I'm wrong but... Check your Convert - Look&Feel page. If the 'Smarten punctuation' box is checked then both dot-dot-dot (3-char) and dot-space-dot-space-dot (5-char) will be converted to a single char ellipsis in the converted file. The visual appearance of the single-char ellipsis will depend on the font face used in the ereader application.

As far as I know Calibre respects the non-breaking-space entity. What I don't know is whether dot-nbsp-dot-nbsp-dot will also be converted to a single char ellipsis if 'Smarten punctuation' is checked.

As far as line-breaking at an ellipsis is concerned, I would think the reading application would decide that.

[DiapDealer]

Originally Posted by jackie_w

What I don't know is whether dot-nbsp-dot-nbsp-dot will also be converted to a single char ellipsis if 'Smarten punctuation' is checked.

Calibre uses SmartyPants to smarten punctuation, so dot-nbsp-dot-nbsp-dot will be left alone by the process.

[dwig]

Originally Posted by Bobosmite

...can Calibre detect this as an ellipsis

probably not as you've inserted spaces between the dots.

and allow the line to break before or after, or is that determined by the reading device?

It's the reading device/app that determines where line wrapping occurs. If you want the reader to be able to break before and/or a true elipsis you need in insert a standard space before and/or after the elipsis.

A true elipsis is a single special character. The three periods style is an antique typewriter kluge. Many apps, including a properly configured MS Word, will replace these with a proper elipsis even if you adjust the letter spacing (aka kerning) between the characters. When you space the periods out with space characters I wouldn't expect any app to do the auto replacement. If you use non-breaking spaces (  in HTML) to space the periods it will flow as if it is a single character, but you need to use standard "breaking" spaces before and/or after if you want the reader to be able to wrap the line at those positions.

Also, does Calibre recognize non-breaking spaces?

Yes, at least in most input formats.

[Bobosmite]

Originally Posted by dwig

If you want the reader to be able to break before and/or a true elipsis you need in insert a standard space before and/or after the elipsis.

Yeah, I had a feeling that might be the case. I'm so used to having full control of the document and going the other direction, to a flat file, is like drawing with ASCII. Once I figure out the conversion logic, it will be much easier.

[ldolse]

Originally Posted by DiapDealer

Calibre uses SmartyPants to smarten punctuation, so dot-nbsp-dot-nbsp-dot will be left alone by the process.

Smarten Punctuation primarily relies on smarty pants, but it also converts ... to ellipsis, and converts -- to em-dash.

[DiapDealer]

Originally Posted by ldolse

Smarten Punctuation primarily relies on smarty pants, but it also converts ... to ellipsis, and converts -- to em-dash.

That's all part of what SmartyPants does. quotes, ellipses, em- en-dash conversion.

I wasn't guessing. I looked at the code for the SmartyPants script that's included with calibre. The only things Smarten punctuation converts to the elipse character is three periods in a row, or period<sp>period<sp>period.

Period-nbsp-period-nbsp-period will escape Smarten punctuation's notice.

[ldolse]

Originally Posted by DiapDealer

That's all part of what SmartyPants does. quotes, ellipses, em- en-dash conversion.

I wasn't guessing. I looked at the code for the SmartyPants script that's included with calibre. The only things Smarten punctuation converts to the elipse character is three periods in a row, or period<sp>period<sp>period.

Period-nbsp-period-nbsp-period will escape Smarten punctuation's notice.

I wasn't guessing either, though I didn't fully get the gist of your response - I missed your point about non-breaking spaces, which is true regardless (although I think the OP asked about nbsp as a separate topic). Last time I checked though Smartypants didn't change three dots in a row, e.g. '...'. I'll readily admit that it's quite possible I'm no longer correct about current behavior due to more recent changes in smartypants. I know my original statement was true at one time in the past because smarten punctuation didn't affect ellipsis or double dashes, to resolve the issue I added and QA'd the extra logic which modifies them. Check the last lines of code here:
http://bazaar.launchpad.net/~kovid/c.../preprocess.py

edit - it's possible you're agreeing with me, but we got mixed up on where the smartypants vs smarten_punctuation code is.

[DiapDealer]

Last time I checked though Smartypants didn't change three dots in a row, e.g. '...'.

Of course it's fully configurable and you can tell it not to convert three consecutive periods to an epllipse if you want, but its default behavior to do so. The two regex patterns it looks for in it's EducateEllipses function is:
\.\.\. and \. \. \.

Spoiler:

def educateEllipses(str):
	"""
	Parameter:  String.
	Returns:    The string, with each instance of "..." translated to
	            an ellipsis HTML entity.

	Example input:  Huh...?
	Example output: Huh…?
	"""

	str = re.sub(r"""\.\.\.""", r"""& #8230;""", str)
	str = re.sub(r"""\. \. \.""", r"""& #8230;""", str)
	return str

Spaces after the "&" added by me so the html entity didn't get eaten by the forum software .

edit - it's possible you're agreeing with me, but we got mixed up on where the smartypants vs smarten_punctuation code is.

That appears to be the case

To be perfectly honest... those last lines of calibre code you linked to have always confused me. SmartyPants already converted '...' and '. . .' to the ellipse html entity... so I'm not sure why it's being done again before doing the substitute_entity call. And the first -- replacement is being done to preserve any html comments, but it seems as if the final "--" substitution would undo all that.

EDIT: never mind that last part about the -- replacement. That appears to be catching any '--' that has a space on either side of it, which would exclude the html comments... but I'm still convinced it's not going to find any occurrances of ' -- ' after the SmartyPants default call.

[ldolse]

Interesting. I didn't realize the logic was already built into smartypants, and I distinctly recall having test cases which weren't converted by smartypants, which was why the extra lines were added - I agree with you that on a quick review of the code they seem redundant, and it's possible something else was going on. I'll dig back into it when I get a chance.