A situation that breaks "smarten punctuation" - Page 3

JSWolf · 05-22-2012, 11:25 PM

Quote:

Originally Posted by dwig

In 0.8.51 it still fails with a closing italics ( or ) tag just before the close quote. See attached examples (original tick-mark quotes in ePub and conversion to MOBI with smart quotes)

In both files, neither show left/right quotes.

dwig · 05-23-2012, 11:02 AM

Quote:

Originally Posted by JSWolf

In both files, neither show left/right quotes.

Check again. I just downloaded the posted MOBI and proofed it in Kindle Preview. It clearly has "typographer's quotes" and the closing quote on the second string of quoted text is incorrect.

The ePub, as stated, uses tick marks instead of true quotes.

Perkin · 05-23-2012, 03:00 PM

From what I can make out, and a few tests, it may be a case of remove the check for an opening quote followed by a space at beginning of the line.

That will fix the problem dwig is having.

Because the way the smarten punctuation works, the code for a start-of-line quote followed with a space and the problem dwig is having, are both caught/caused by the same regex, and I can't (at moment) see a way to distinguish the two - because codewise they are the same.

So it may be better and more easily fixable afterward to remove that check so dwig' problem would never have happened.

Then HarryT (and anyone else) would be able to find the start-of-line quote + space in sigil or other editor.
That I think will be easier to fix than trying to find the midline leftquote, (or it might not).

What are your opinions?

JSWolf · 05-23-2012, 06:49 PM

Quote:

Originally Posted by dwig

Check again. I just downloaded the posted MOBI and proofed it in Kindle Preview. It clearly has "typographer's quotes" and the closing quote on the second string of quoted text is incorrect.

The ePub, as stated, uses tick marks instead of true quotes.

My mistake. Yes, the Mobi does have smart quotes. It's that the default font was so small, it was hard to notice. Once I upped the font size, it was easy to see.

ldolse · 05-24-2012, 03:36 AM

A fix was checked in for this in the past day or two - basically the two problem regexes have been commented out. This issue is that smarten punctation doesn't have any concept of a 'sentence'. It 'tokenizes' the html by breaking everything into either an html tag or a piece of text - the pieces of text get passed to the regexes.

Those two regexes that are causing the problem are using line start/line end regex meta-characters--^$--Since there is no guarantee that any piece of text the regex sees will actually be the start or end of the line we had to disable those patterns. And in the case of style tags like ,,,, etc a sentence would indeed be broken into multiple pieces.

Basically the take-away is there won't be any way to smarten some 19th/early 20th century books that use odd quoting patterns like some of the Holmes books, at least not within the context of how smarten works.

Bug for reference:
https://bugs.launchpad.net/bugs/998900

05-24-2012, 03:36 AM	#35
ldolse Wizard Posts: 1,337 Karma: 123457 Join Date: Apr 2009 Location: Malaysia Device: PRS-650, iPhone	A fix was checked in for this in the past day or two - basically the two problem regexes have been commented out. This issue is that smarten punctation doesn't have any concept of a 'sentence'. It 'tokenizes' the html by breaking everything into either an html tag or a piece of text - the pieces of text get passed to the regexes. Those two regexes that are causing the problem are using line start/line end regex meta-characters--^$--Since there is no guarantee that any piece of text the regex sees will actually be the start or end of the line we had to disable those patterns. And in the case of style tags like <i>,<b>,<em>,<span>, etc a sentence would indeed be broken into multiple pieces. Basically the take-away is there won't be any way to smarten some 19th/early 20th century books that use odd quoting patterns like some of the Holmes books, at least not within the context of how smarten works. Bug for reference: https://bugs.launchpad.net/bugs/998900 Last edited by ldolse; 05-24-2012 at 03:44 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Smarten punctuation slows conversion	kakkalla	Calibre	22	01-20-2011 08:22 AM
Configuration of "Smarten Punctuation"	Korben Dallas	Calibre	2	12-31-2010 08:12 AM
Simpler Way to Smarten Punctuation	Rand Brittain	Calibre	3	10-10-2010 09:16 PM
Thanks for new 'Smarten Punctuation' feature	jackie_w	Calibre	1	09-21-2010 03:53 PM
"The Situation" by Jeff VanderMeer - free download	Kingston	Deals and Resources (No Self-Promotion or Affiliate Links)	10	03-18-2008 12:39 AM

05-23-2012, 03:00 PM	#33
Perkin Guru Posts: 657 Karma: 64171 Join Date: Sep 2010 Location: Kent, England, Sol 3, ZZ9 plural Z Alpha Device: Sony PRS-300, Kobo Aura HD, iPad (Marvin)	From what I can make out, and a few tests, it may be a case of remove the check for an opening quote followed by a space at beginning of the line. That will fix the problem dwig is having. Because the way the smarten punctuation works, the code for a start-of-line quote followed with a space and the problem dwig is having, are both caught/caused by the same regex, and I can't (at moment) see a way to distinguish the two - because codewise they are the same. So it may be better and more easily fixable afterward to remove that check so dwig' problem would never have happened. Then HarryT (and anyone else) would be able to find the start-of-line quote + space in sigil or other editor. That I think will be easier to fix than trying to find the midline leftquote, (or it might not). What are your opinions?

Advert

Advert