False paragraph breaks & RegEx

[ColMac]

Wonder if someone can help an absolute beginner.

I have a number of books that contain many false paragraph breaks part way through a sentence.

An example of the type of code that appears is

</p>

<p class="calibre7">

This of course is the valid code for a correctly placed paragraph break in this book.

It normally appears with no space after the last word, and no space before the next word, and in most (I'm reluctant to say all) cases the last character before the error will be lower case. However, the first character of the following word may well be upper case when a name is used for example.

Would there be a way to check for that code appearing (and Calibre 7 would presumably need to be a variable) when not following the common punctuation marks?

Thanks in hope, as this is way beyond my skills.

Colin

[JSWolf]

Is this a Calibre conversion from a PDF? Can you share some of the code?

[ColMac]

Originally Posted by JSWolf

Is this a Calibre conversion from a PDF? Can you share some of the code?

Unfortunately I have no idea. I am looking at them as epubs in Calibre.

I have just manually edited the few errors I spotted in the book I'm reading, so I don't have any live examples to show, and a search obviously is no help.

I should be able to find an example in a few days in my next book

[gbm]

Look in the sticky thread--Saved Search/Regex Functions--at the top of this editor forum, I believe the first post will have the help you need.



bernie

Originally Posted by ColMac

Wonder if someone can help an absolute beginner.

I have a number of books that contain many false paragraph breaks part way through a sentence.

An example of the type of code that appears is

</p>

<p class="calibre7">

This of course is the valid code for a correctly placed paragraph break in this book.

It normally appears with no space after the last word, and no space before the next word, and in most (I'm reluctant to say all) cases the last character before the error will be lower case. However, the first character of the following word may well be upper case when a name is used for example.

Would there be a way to check for that code appearing (and Calibre 7 would presumably need to be a variable) when not following the common punctuation marks?

Thanks in hope, as this is way beyond my skills.

Colin

[JSWolf]

Originally Posted by ColMac

Unfortunately I have no idea. I am looking at them as epubs in Calibre.

I have just manually edited the few errors I spotted in the book I'm reading, so I don't have any live examples to show, and a search obviously is no help.

I should be able to find an example in a few days in my next book

Where did this ePub come from?

[Sarmat89]

First, try this in replace all mode:

(?<=[\p{Ll},;]) *</p>\s*<p class="[^"]*">(?=\p{Ll})

, replacing this with a space.
Then, remove (?=\p{Ll}) part, and do manual replacements troughout the file.
Then, restore that part, and remove the (?<=....) one; continue replacement with manual confirmations.

[Tex2002ans]

Originally Posted by ColMac

Wonder if someone can help an absolute beginner.

I have a number of books that contain many false paragraph breaks part way through a sentence.

An example of the type of code that appears is

</p>

<p class="calibre7">

This of course is the valid code for a correctly placed paragraph break in this book.

It normally appears with no space after the last word, and no space before the next word, and in most (I'm reluctant to say all) cases the last character before the error will be lower case. However, the first character of the following word may well be upper case when a name is used for example.

I have explained this exact "broken paragraphs" question many times over the years.

Here are 4 such topics where I go step-by-step and break down the Regular Expressions:

• 2021: "Regex examples"
  
  • (Especially my Post #689+.)
• 2020: "regex newbie search end of string char problem"
• 2016: "Delete paragraphs in scanned books (S & R with regexes)"

I even discussed it way back in:

• 2014: "How can I fix it when every line is a paragraph?"

(Of course, my newer methods are better and fix more things, but those regex are still useful to see/learn from.)

[enuddleyarbl]

Originally Posted by Tex2002ans

I have explained this exact "broken paragraphs" question many times over the years.

Here are 4 such topics where I go step-by-step and break down the Regular Expressions:
[LIST][*] 2021: "Regex examples"

• (Especially my Post #689+.)

...

From that 2021 post:

Search: -</p>\s+<p>
Replace: <--- (Completely blank)

and:

Search: ([^>”\?\!\.])</p>\s+<p>
Replace: \1 <---- (There's a space after the '1')

and:

Search: <p>[a-z]
Replace: <---- (BLANK. Only use for FINDING, NOT REPLACING.)

I don't know if Sigil is different from Calibre in this regard (the post is in the Sigil forum), but if the last search (for lowercase starting paragraphs) is saved in Calibre, make sure to check the "Case Sensitive" box.

Also, a question for the first ("hyphen") rule. Most of the books I edit have a tendency to end interrupted paragraphs with a dash:

<p>"Here I am wal-</p>
<p>The monster leaped out and ate my face.</p>

Stylistically, should the dash be replaced with something else (to signify an interruption)? Or, just leave it?

EDIT: And, wouldn't the first rule be covered by the second? Are you just separating them to pull the hyphen issues out of the mass?

[Quoth]

Should be an em dash with no space but closing quote for cut-off dialogue. Depends on style guide. We use en dash surrounded by spaces when it's parenthetical and like a comma the 2nd one is omitted at the end of a sentence unlike (actual brackets). Many USA style guides use em dash without spaces inside a sentence for parenthesis. An actual hyphen shouldn't be an en or em dash and should only have spaces either side if it's a range.
Actual hyphens come in at least three kinds:
1) A word broken at a suitable point to wrap at the margin
2) A word that is commonly using a hyphen. Ones that are not the same vowel repeated drop the hyphen over the years, so today used to be to-day.
3) Suffixed words, as in "His face was fish-like".

Obviously in removing excess hyphens 2 & 3 must be left and the era of writing considered to see if hyphens of type 2 are valid.

[Tex2002ans]

Originally Posted by enuddleyarbl

I don't know if Sigil is different from Calibre in this regard (the post is in the Sigil forum), but if the last search (for lowercase starting paragraphs) is saved in Calibre, make sure to check the "Case Sensitive" box.

Yes, by default, Calibre's Regex is case insensitive (why that is, I'm not sure).

In Calibre, make you you have that "Case Sensitive" box checked.

In Sigil, all you have to do is make sure you are in "Regex" mode.

Originally Posted by enuddleyarbl

Also, a question for the first ("hyphen") rule. Most of the books I edit have a tendency to end interrupted paragraphs with a dash:

<p>"Here I am wal-</p>
<p>The monster leaped out and ate my face.</p>

Stylistically, should the dash be replaced with something else (to signify an interruption)? Or, just leave it?

When conversations get cut off, the proper character to use is an:

• — = EM DASH
  
  • (It is about the size of an 'm'.)

I recommend reading this great article in Wikipedia showing different examples:

• Wikipedia.org: "Dash" > Interruption of a Speaker

and my many writings over the years:

• 2018: /r/writing: "Punctuation is important too"
• 2021: /r/PubTips: "[PubQ] Quick question about the em dash."
• 2020: "Punctuation Rules of Thumb?"
  
  • I cover all the types of dashes + ellipses + quotation marks + primes (feet/inches).

- - -

In the case where you have a wrong/bad interruption:

<p>"Here I am wal-</p>
<p>The monster leaped out and ate my face.</p>

you would use this Regex:

Find: -</p>
Replace: —</p>

(Replace a HYPHEN at the end of a paragraph with an EM DASH.)

That will get you the correct:

<p>"Here I am wal—</p>
<p>The monster leaped out and ate my face.</p>

Note: DO NOT ever do a "Replace All" though, you'd have to decide these on a case-by-case basis.

- - -

Side Note: Actually, your example is off. Just because your dialogue got interrupted... you'd still need the close quote:

<p>“Here I am wal—”</p>
<p>The monster leaped out and ate my face.</p>

I'll let you figure out the Replace needed for that.