Creating smart quotes in Sigil?

[MacEachaidh]

Folks, I'm fairly new to using Sigil properly - I've just started trudging up the learning curve of coding - so pardon me for asking this. I thought it would have already been asked and answered in the past, but I couldn't find it with a search, so:

Is there a straightforward way to convert plain ASCII quotes to "smart" or "typographic" quotes?

I can do a manual Find and Replace, searching for opening quotes, and then repeat it for closing quotes, and then apostrophes ... but is there a shorter route?

Thanks for any help.

[Toxaris]

You might be able to do it with some good RegEx's... Personally I do it in Word before creating the ePUB.

[MacEachaidh]

Yes I do, too, Toxaris, thanks (Except I use WordPerfect - I dislike Word And ePubs I'm creating from scratch I put through Jutoh, rather than Sigil.)

But that's not why I was asking. I also have some fairly shoddily-formatted commercial epubs that give me the irrits, which I want to tidy up. Most of what I need to do I've nutted out, but not the quotes.

Anyone got an answer for this, please?

[alecE]

I'm incompetent in Regex, so I have a fairly laborious procedure, which gets done in Notepad++ after any necessary scanning/OCR processes and cleaning-up line breaks:
(I prefer double quotes for speech, single quotes for abbreviations, apostrophes etc)
-insert <p> at start of first line;
-change all carriage-return/new-lines to </p>\r\n\r\n<p>;
- insert </p> at end of last line;
-change all <p>" to <p>&ldquo;
-change all "</p> to &rdquo;</p>;
-change all ^"space to ^&rdquo;space (where ^ may be stop, comma, query or bang);
-change all ^space" to ^space&ldquo; (where ^ may be stop, comma, query, bang, colon or semi-colon);
-by now the number of instances of spacequote and quotespace should be sufficiently few to permit individual search/replace with double or single quotes as required - several passes may be required.
- run through, tracking down the last few instances of quotes, then do a mega replace of single quotes with &rsquo; for the abbreviations.
-sort out the ndashes and ellipses;
Tedious, but it gets me there in the end - a typical SF book of 8 signatures will take me 3 to 6 hours to read, correct and edit, i.e. from OCR-produced text file through to Sigil-ready html.

I find it pays to use named entities - it's particularly helpful when converting a text that has single quotes for direct speech into double-quoted speech marks. I suspect there are various magic formulas in Regex which could do the job as well. If I can find a few spare brain cells one day, I may try going down that route.
Bottom line - no easy solution

[DiapDealer]

But that's not why I was asking. I also have some fairly shoddily-formatted commercial epubs that give me the irrits, which I want to tidy up. Most of what I need to do I've nutted out, but not the quotes.

SmartyPants (originally written in Perl and ported to Python). It's what calibre uses. Nothing's going to be perfect for this sort of thing but it does very well. Areas where it fails are fairly easy to clean up afterwards. I run xhtml through it before ever opening in Sigil. It will create them as HTML entities though.

[mmat1]

Originally Posted by MacEachaidh

Is there a straightforward way to convert plain ASCII quotes to "smart" or "typographic" quotes?

How would you describe the text, where an opening quote is? I guess, there's some white space or a <p> before and some letters after a opening quote.

In regex this should be

(<p>|\s)"(\w)

replaced with

\1“\2

. For closing quotes:

(\S)"(\s|</p>)

.

I hope i got this regex right, i'm still on a learning curve as well . But in general it should work in most (maybe not all) cases.

[JSWolf]

Originally Posted by DiapDealer

SmartyPants (originally written in Perl and ported to Python). It's what calibre uses. Nothing's going to be perfect for this sort of thing but it does very well. Areas where it fails are fairly easy to clean up afterwards. I run xhtml through it before ever opening in Sigil. It will create them as HTML entities though.

How do you use SmartyPants? Do I have to write Python code to use it?

[DiapDealer]

How do you use SmartyPants? Do I have to write Python code to use it?

At least a little bit. It's designed to be imported as a python module.

Then your code can be as simple as:
Read in the html file
Pass the html data to the smartyPants function
Write the data it returns to a new file.

If you give me a little time, I might be able to post a pretty simple working sample (as long as you know your way around command-line stuff).

[JSWolf]

Originally Posted by DiapDealer

At least a little bit. It's designed to be imported as a python module.

Then your code can be as simple as:
Read in the html file
Pass the html data to the smartyPants function
Write the data it returns to a new file.

If you give me a little time, I might be able to post a pretty simple working sample (as long as you know your way around command-line stuff).

Thanks. I'm very familiar with the command-line. I've been using a command line before a GUI was a thought in Xerox's mind.

[DiapDealer]

OK. I'm assuming you already have Python installed and working.

First get the latest smartypants.py script from here (bottom of the page). Either copy and paste the text from the browser or right-click and "save link as." Just make sure you end up with a file named "smartypants.py". (Alternatively, you can install smartypants.py as a library module in your Python installation if you know how to do that.) I tend to use the smartypants.py script included in the calibre source.

Download and unzip the smarty-wrapper.py script from the zip-file I've attached to this post (the forum won't let me attach a .py file).

Now... put the (x)html file you want to "smarten" in a folder along with the smartypants.py file and the smarty-wrapper.py file (you can forget about the smartypants.py script if you've already installed it as a module).

From a command prompt... cd to the folder where all three files are located and issue the following command:

python smarty-wrapper.py infile.html outfile.html

Of course make sure to change infile.html to whatever the name of the file you're trying to modify really is. And change outfile.html to whatever you want your resulting new file to be called.

You can read the extensive documentation that's included in the smartypants.py script in order to modify its behavior. Just know that the default action is to smarten quotes (double and single), change -- to an mdash, and to change three consecutive periods ... into an ellipsis …

[Serpentine]

I have some regex saved here which does a better job than smarty most of the time (or at least I've not run into bad cases). It needs some translation from GJSoft to PCRE - but I can do that a bit later if you are still looking for a way to do it.

Only thing I can think of is that it doesn't like multi-paragraph quotes. There's an easy way to find those tho.

[DiapDealer]

Originally Posted by Serpentine

I have some regex saved here which does a better job than smarty most of the time (or at least I've not run into bad cases). It needs some translation from GJSoft to PCRE - but I can do that a bit later if you are still looking for a way to do it.

Only thing I can think of is that it doesn't like multi-paragraph quotes. There's an easy way to find those tho.

Yes, situations when there isn't always a closing quote is almost always going to cause issues with any automated script. Also apostrophes at the beginning of a word (like 'tis) are problematic for smarty as well (since there's no programmatic way to determine that it's just not an opening single quote). But overall, I've found it to do a much better job (not to mention quicker and easier) with double/single quotes, mdashes, ellipses than any regex I've come across (or come up with).

If you get it ported to PCRE, I'd certainly like to see it. It's always preferable to stay within Sigil if I possibly can. But whatever it is, it's going to have to work in code view (which means excluding the quotes around tag attributes) since Find & Replace in BV is just not practical/advisable with Sigil.

[JSWolf]

Thank you for smarty-wrapper.py.

[kiwidude]

In the Modify ePub plugin for calibre I not so long ago added the option to just do "Smarten Punctuation" which will just run calibre's smarty pants stuff against the html files in the ePub (and do nothing else if you so choose it).

[DiapDealer]

Originally Posted by kiwidude

In the Modify ePub plugin for calibre I not so long ago added the option to just do "Smarten Punctuation" which will just run calibre's smarty pants stuff against the html files in the ePub (and do nothing else if you so choose it).

Brilliant! I wasn't aware of that. Thanks!
Out of curiosity, will that leave the new smartened punctuation as HTML entities or does it change them to their unicode counterparts like the normal calibre conversion option seems to?

EDIT: Nevermind, I see it changes them to their unicode equivalents (except for the elipsis)... which is what I want anyway. Thanks again.