Problems with regex and text flag

[jwes]

I was using this regex: "(.*?)"
the quotes are part of the regex
on this p

<p class="msonormal3">“Ah,” Pen Rel said again, and inclined his head. "Mostly, it is a matter of temperature control.<br class="calibre13"/>How much simpler, after all, to let the wandering air take the heat away than to condition the dock entire.”</p>

and the found text was:

"Mostly, it is a matter of temperature control.<br class="

Also ^ and $ don't work how I would expect. They seem to match only after and before newlines in text rather than the start and end of paragraph text.

I also find it very difficult to distinguish between “, ", and ” in the find and replace boxes.

[DNSB]

One item is that you have a mix of curly quotes around the “Ah,” for example) and straight quotes around the "Mostly, it is a matter of temperature control.<br class=" for instance. If I modify the quotes to curly quotes for the content:

<p class="msonormal3">“Ah,” Pen Rel said again, and inclined his head. “Mostly, it is a matter of temperature control.<br class="calibre13"/>How much simpler, after all, to let the wandering air take the heat away than to condition the dock entire.”</p>

and then run the regex as “.*?”, it finds two items, “Ah,” and “Mostly, it is a matter of temperature control.<br class="calibre13"/>How much simpler, after all, to let the wandering air take the heat away than to condition the dock entire.” which is more or less correct.

You might want to try smartening the punctuation.

BTW, using a <br> to break in the middle of a paragraph is a bad idea. Let the renderer break the lines where it needs to.

[jwes]

Originally Posted by DNSB

One item is that you have a mix of curly quotes around the “Ah,” for example) and straight quotes around the "Mostly, it is a matter of temperature control.<br class=" for instance. If I modify the quotes to curly quotes for the content:

<p class="msonormal3">“Ah,” Pen Rel said again, and inclined his head. “Mostly, it is a matter of temperature control.<br class="calibre13"/>How much simpler, after all, to let the wandering air take the heat away than to condition the dock entire.”</p>

and then run the regex as “.*?”, it finds two items, “Ah,” and “Mostly, it is a matter of temperature control.<br class="calibre13"/>How much simpler, after all, to let the wandering air take the heat away than to condition the dock entire.” which is more or less correct.

You might want to try smartening the punctuation.

That's what I was doing. They were all straight quotes when I started.

BTW, using a <br> to break in the middle of a paragraph is a bad idea. Let the renderer break the lines where it needs to.

This is a Calibre conversion, and Calibre does things like that.

[DNSB]

Originally Posted by DNSB

You might want to try smartening the punctuation.

BTW, using a <br> to break in the middle of a paragraph is a bad idea. Let the renderer break the lines where it needs to.

Originally Posted by jwes

That's what I was doing. They were all straight quotes when I started.

This is a Calibre conversion, and Calibre does things like that.

Not sure if that is a calibre artifact, IMHO, it's a lot more likely that that was in the original document. Since you are editing the file, it's not that big a deal to remove those embedded <br>s.

For the most part, my first pass at smartening punctuation is using the Modify Epub calibre plugin.

[Turtle91]

Or, since you are using Sigil, the Smarten Punctuation plug-in.

[jwes]

Originally Posted by Turtle91

Or, since you are using Sigil, the Smarten Punctuation plug-in.

Thanks, that is a good suggestion, but I'm mostly concerned that a regex replace with the text flag set can break the html.

[KevinH]

The text flag is nothing more than the automatic prepending of the following regex:

static const QString REGEX_OPTION_TEXT_ONLY = "<[^<>]*>(*SKIP)(*F)|";

which can be overruled by later regex you add.

If should only match text outside of < > chars unless overruled by later regex.

Try turning off text and prepending it yourself to try to see what is interfering.

[KevinH]

Ah! Your use of ? to indicate Minimal Match (non-greedy) will actually *invert* the Greediness of the Minimal Match Regex Option. The Text flag needs its part to default to greedy.

Try replacing the smart quote at the end of "entire." with a normal quote and then remove the ? that toggles the initial Text only regex to be nongreedy.

ie. use "(.*)"

and then make sure the Minimal Match and DotAll are both set in the Regex options and make sure the Text box is checked.

That seems to work.

But using a real parser or the Smarten plugin is probably your best bet here as corner cases will be found.

If you do decide to use Search and replace and regex, you should first do a Dry Run using Shift key on the Count (#) button or better yet use Shift on the Replace All button to see a complete table of the potential replacements and allow you to remove any corner cases (filter those changes out) before proceeding.

Both Dry Run Replace All and Filtered Replace All are newer Find and Replace tools that really help in situations where unspecified corner cases may exist.

Give them a try.

Additionally, making a Checkpoint would not hurt either.