Need help with regex

[jordy1955]

Hi,
Firstly let me say that I am a very rudimentary user of regex. Most of it is beyond my comprehension.

I have some eBooks that were clearly produced by less than spectacular OCR software.

Accordingly, the formatting ranges from quite good to really bad.

One of the main problems is line breaks in the wrong places (eg in the middle of a sentence), making the text very difficult to follow.

In F&R I have used this "[a-z]</p><p class="calibre_1">" - or similar - to quite successfully find these instances, but the problem is that the entirety of the matched regex is selected and I cannot for the life of me work out how to get the replace function to disregard the [a-z] component of the result in order to avoid what can be hundreds of manual interventions to fix all the errors.

Any assistance is gratefully accepted.

thanks

Paul

[Sarmat89]

Use
(?<=\p{Ll})</p>\s*<p class="...">

[jordy1955]

Originally Posted by Sarmat89

Use
(?<=\p{Ll})</p>\s*<p class="...">

Thanks for this, I have no idea what it does but I'm guessing that somehow it ignores the text captured BEFORE the '</p><p class="calibre_1">'.


Thankyou.

[jordy1955]

Originally Posted by Sarmat89

Use
(?<=\p{Ll})</p>\s*<p class="...">

OK I tried this (copy and paste from this thread) in the find field and it does not find anything whereas [a-z]</p><p class="calibre_1"> does...

Is there something I'm missing?

[davidfor]

Originally Posted by Sarmat89

Use
(?<=\p{Ll})</p>\s*<p class="...">

That doesn't work for me and I can't work put what the look behind is supposed to do.

I use:

([\w,—])</p>\s*<p\s*[^>]*?>([\w])

with the replace of:

\1 \2

It doesn't catch everything (it probably should have the other types of dash) but it catches most.

[jordy1955]

This is what my query returns.

I need to exclude the Single char - in this case the "E" - either in the search result or exclude it in the replace function.

[jordy1955]

Originally Posted by davidfor

That doesn't work for me and I can't work put what the look behind is supposed to do.

I use:

([\w,—])</p>\s*<p\s*[^>]*?>([\w])

with the replace of:

\1 \2

It doesn't catch everything (it probably should have the other types of dash) but it catches most.

This works, BUT, it also returns the 1st char of the following word - see image

How then do I exclude the unwanted chars in the replace field? i've got no idea what the \1 \2 means

[jordy1955]

Originally Posted by jordy1955

This works, BUT, it also returns the 1st char of the following word - see image

How then do I exclude the unwanted chars in the replace field? i've got no idea what the \1 \2 means

I did say that I'm pretty basic in my understanding of regex... I just realised what the \1 \2 does. Tested it and it works beautifully.

thankyou so much. You have saved me hours of manual intervention and frustration

[davidfor]

Originally Posted by jordy1955

I did say that I'm pretty basic in my understanding of regex... I just realised what the \1 \2 does. Tested it and it works beautifully.

thankyou so much. You have saved me hours of manual intervention and frustration

The parentheses are "capture groups". And then "\1" is the first group, "\2" is the second and so on.

Another I have used recently was:

([[:lower:]])\s*</p>\s*<p>\s*([[:lower:]])

That is for when the paragraph ends in a lower case letter and the next starts with a lower case letter. Maybe with the spaces. For that, I am sure it is a paragraph that has been split. For the first one, I generally look at them to check what is actually intended.

And this one doesn't cater for the class. If I am doing this amount of fixing, I remove the class for the normal paragraph. If there are any left, it probably means there is other formatting that I probably don't want to lose.

[enuddleyarbl]

Originally Posted by davidfor

...I use:

([\w,—])</p>\s*<p\s*[^>]*?>([\w])

with the replace of:

\1 \2

It doesn't catch everything (it probably should have the other types of dash) but it catches most.

It might be a dependency of the editor, but in Calibre's you'd have to escape the "/" in "</p>". So, it would be:

([\w,—])<\/p>\s*<p\s*[^>]*?>([\w])

BTW: it's an amazing thing, but I came here looking for the exact same thing just now.

[enuddleyarbl]

@jordy1955: I've been using

https://regex101.com/

to try various regex things and see what they do. It's been a lot of help.

One thing to note, though, the replacement character they use there is a $ instead of the \ used in Calibre's editor. So, if you wanted to test davidfor's replacement string of:

\1 \2

there, you'd have to use:

$1 $2

[davidfor]

Originally Posted by DaveLessnau

It might be a dependency of the editor, but in Calibre's you'd have to escape the "/" in "</p>". So, it would be:

([\w,—])<\/p>\s*<p\s*[^>]*?>([\w])

No, its a Regex version thing. Calibre uses Python's regex. That doesn't need the forward slash to be escaped. For others, such as PCRE, it will need to be escaped.

Originally Posted by DaveLessnau

@jordy1955: I've been using

https://regex101.com/

to try various regex things and see what they do. It's been a lot of help.

One thing to note, though, the replacement character they use there is a $ instead of the \ used in Calibre's editor. So, if you wanted to test davidfor's replacement string of:

\1 \2

there, you'd have to use:

$1 $2

Again, it is the regex version. You should choose Python under "Flavor" to test calibre regex's.

[jordy1955]

Awesome stuff guys. Just ran it on a book and - once I got my head around it properly - I completed the editing and re-formatting in about 1hr - about 4 hours less than it usually takes me.
I'll get much quicker with practice but this is great.

Again, thanks SO MUCH.

Paul

[gbm]

Link to the calibre Quick reference for regexp syntax.

bernie

Originally Posted by jordy1955

Thanks Guys, Awesome!
Just fixed a book in about 1hr where it would have taken me about 5hrs previously - fixing one instance at a time...

I'll get quicker after some practice.
Again, thank you SO MUCH!

Paul

[enuddleyarbl]

Originally Posted by davidfor

No, its a Regex version thing. Calibre uses Python's regex. That doesn't need the forward slash to be escaped. For others, such as PCRE, it will need to be escaped.

Again, it is the regex version. You should choose Python under "Flavor" to test calibre regex's.

I just double-checked, and you're right. I didn't know I could change the behavior of that regex101 site by changing the flavor and I think that's where I got the idea the Calibre editor didn't like those / symbols without escaping.

This has been a productive thread for me: I found a much better search/replace for fixing badly split paragraphs, I learned that I could change the behavior of the regex101 site to match Calibre's editor, and some of the search strings I use will be easier now that I won't have to escape the / character. Thanks.