How can I fix it when every line is a paragraph?

[eschwartz]

Originally Posted by Nyssa

Yep. These were gifted to me awhile ago and I'm just now ready to read them.

They are brilliant and tremendously funny, among my all-time favorites.

[Nyssa]

Originally Posted by Nyssa

Oh! Sorry, I thought I could choose between options. I didn't realize they were steps... Hyphens aren't a problem, so I figured I didn't need Regex #2 or its variant.

Thank you.

So Regex 1 was the only one that ran successfully. I got the "0 Occurance" message with both Regexes 2 & 3.

I even tried the variant.

[eschwartz]

Mystery solved.

Tex2002ans, your regexes don't account for classes, how could you!

Fixed:

Regex #2:

Search: -</p>\s+<p( [^<>]+)?>
Replace:

Explanation: What this will do is remove hyphens at the very end of the "paragraph", and combine it with the next line.

Side Note: I use the above regex on a one-by-one, case-by-case basis, because many "soft hyphens" in the PDF aren't actually a part of the word.

Example:

<p>Blah blah blah govern-</p>
<p>ment.</p>

<p>Blah blah blah government.</p>

Regex #2 (Variant):

Search: -</p>\s+<p( [^<>]+)?>
Replace: -

Note: I don't use this one, although if there are TONS of hyphens at the end of each line, it might be best to do it this way, and take care of the hyphen situation on your own at a later step. I personally prefer to use the Spell Check Tool, and search for a single hyphen by itself: '-'. This will give you a list of every single word with a hyphen in it. Then I can check for + fix mistakes there much more quickly.

Example:

<p>Blah blah blah govern-</p>
<p>ment.</p>

<p>Blah blah blah govern-ment.</p>

Regex #3:

Search: ([^>”\?\!\.])</p>\s+<p( [^<>]+)?>
Replace: \1

Explanation: What this Regex will do, is search for a paragraph that DOES NOT end in a "greater than sign", "right double quote", "question mark", "exclamation point", or "period". It will then combine it with the next paragraph.

Note: There is a space after the "\1".

Example:

<p>Susie said</p>
<p>that she was going to jump over a tree.</p>
<p>She also said,</p>
<p>that this was just a sample.</p>

<p>Susie said that she was going to jump over a tree.</p>
<p>She also said, that this was just a sample.</p>

[theducks]

Another trailing or leading hyphen gotcha is the emdash substitute (--).

IMHO replace those with a emdash as step 0

I would love a S&R with an additional alternate replace in case of step 2 is a true hyphenated word split between lines where you can select the Normal Replace (merge lines with no space) and alternate replace ( just Merge). Thus saving 2 passes. The other (third) choice is still Find (Skip)

[theducks]

Originally Posted by eschwartz

Mystery solved.

Tex2002ans, your regexes don't account for classes, how could you!

Fixed:

and you may need a <p class= <span
version

</span></p>\s+<p class="normal"><span class="override1">

The code tips we toss out are generic solutions that may need to be honed to fit a specific case. Never fully trust a bulk (replace all) change.

The Span case is very fragile (and destructive) if there are additional mid paragraph spans. Use extra care, work on a single section until valid (and frequent saves every time you show no errors in the Preview. Do not rly on auto fix solutions

[eschwartz]

Difference is, classes are expected by default, spans are not.

The generic solution caters to common cases, which this is. spans, not so much.

[Tex2002ans]

Originally Posted by Nyssa

Oh! Sorry, I thought I could choose between options. I didn't realize they were steps... Hyphens aren't a problem, so I figured I didn't need Regex #2 or its variant.

Ahhh, sorry, I am used to typing in the technical sections of MobileRead, and I make certain assumptions about the general knowledge that the user has (for example, already knowing what Regex is, and how to use/read it).

Next time, I will have to be even more specific. (Typically, I color code all the sections of the Regex too!).

And yes, they have to be run #1, then #2 (or its variant, depending on if you want to do hyphenation fixes now or later), then #3.

I have a lot more I Regex I recommend after that, although it might get a little too technical in here. (And it does take forever to write these things).

Side Note: I convert a ton of non-fiction economics books from PDF -> EPUB, and deal with cleaning up a lot of crap. I use those regex to mostly piece together lines/paragraphs that broke across pages, or were OCRed incorrectly.

Originally Posted by eschwartz

Tex2002ans, your regexes don't account for classes, how could you!

Well they are the exact Regex that I use. By the time I run those, I already have stripped/clean source, not ones riddled with "calibre#", and who knows what other classes! Perhaps I was just subtly recommending that all those classes be cleaned up as well!

I personally wouldn't recommend the one that handles every <p class="">.... because who knows what a given calibre# associates to ("calibre2" could be your typical paragraph, but "calibre3" could be a blockquote (extra margin on the left), "calibre4" could be right alignment, "calibre5" could be small font, etc. etc.).

Example: That "all classes" Regex would break in these cases. Instead of using a <blockquote> tag, the book might have used something along these lines:

<p>This is a quote from Tex2002ans</p>
<p class="blockquote1">This is a sample blockquote sentence.</p>
<p class="blockquote2">This is some more sentences.</p>
<p class="blockquote2">And this is the end.</p>
<p>Continue with the story.</p>

or the book might have had poetry:

<p class="poem">This is a poem,</p>
<p class="poem2">that is written by Tex.</p>
<p class="poem">This is a poem,</p>
<p class="poem2">that will break the Regex.</p>

So, long story short, clean up the classes first, then run the nice Regex once you know everything you are piecing together is actually a broken paragraph!

Originally Posted by theducks

The code tips we toss out are generic solutions that may need to be honed to fit a specific case. Never fully trust a bulk (replace all) change.

Oh yeah, definitely don't press "Replace All" while running a Regex, unless you know EXACTLY what it is doing. Even then, be sure to thoroughly test them.

Even though I trust Regex #2 and Regex #3 with my life, I still have them in my Sigil's Saved Searches under the heading, "One at a Time".

[Nyssa]

Thank you all!

The books did use "< span >" quite a bit (should anyone else have a similar issue).

As for my case, between this thread, experimenting with the "Edit Book" function and some fancy google-foo by my husband, the issue is solved.

[eschwartz]

Originally Posted by Tex2002ans

Well they are the exact Regex that I use. By the time I run those, I already have stripped/clean source, not ones riddled with "calibre#", and who knows what other classes! Perhaps I was just subtly recommending that all those classes be cleaned up as well!

I personally wouldn't recommend the one that handles every <p class="">.... because who knows what a given calibre# associates to ("calibre2" could be your typical paragraph, but "calibre3" could be a blockquote (extra margin on the left), "calibre4" could be right alignment, "calibre5" could be small font, etc. etc.).

Example: That "all classes" Regex would break in these cases. Instead of using a <blockquote> tag, the book might have used something along these lines:

<p>This is a quote from Tex2002ans</p>
<p class="blockquote1">This is a sample blockquote sentence.</p>
<p class="blockquote2">This is some more sentences.</p>
<p class="blockquote2">And this is the end.</p>
<p>Continue with the story.</p>

or the book might have had poetry:

<p class="poem">This is a poem,</p>
<p class="poem2">that is written by Tex.</p>
<p class="poem">This is a poem,</p>
<p class="poem2">that will break the Regex.</p>

So, long story short, clean up the classes first, then run the nice Regex once you know everything you are piecing together is actually a broken paragraph!

Well, except for poems, such cases would be correctly joined into one whatever-it-is containing the first p's style.

And that is a rationale for double-checking each one, not for writing a regex that doesn't handle lots of stuff.

Alternatively, you can always do it your way... assuming you add another step for clearing up the classes.


FWIW, I agree that my first step would be to clean up the styles, tossing out everything that wasn't very deliberate.

[BetterRed]

Originally Posted by Nyssa

First of all - " mufti " is a new term for me. I know them as "civies".

English borrows 'mufti' from Urdu, which borrows it from Arabic (means scholar). In colonial times many of the officers the Indian Army were Moghuls, hence they spoke Urdu.

Are 'borrowed' words ever returned? I know of none. Linguists are as bad as comp scientists at butchering my language

I guessed the regex aficionado troops would turn up to do battle. Hubby's are handy too.

BR

[JSWolf]

Originally Posted by Nyssa

I have a book that for some reason has every line, regardless of punctuation, listed as a paragraph. (Please see image)

Is there a way, using the editor, that I can remove all of the paragraph html tags at once ?

I'm practically reading the book as I'm trying to correct it (not an enjoyable experience) and it seems all of the books in this particular group were formated the same way.

Attachment 132813

There is no way to fix this eBook without having a paper copy in front of you so you know what goes where. And you have to check every space and every punctuation mark as well to make sure you have it correct.

My suggestion is to contact the shop you bought it from and try to get your money back. It's a major botch job.

[eschwartz]

Originally Posted by JSWolf

There is no way to fix this eBook without having a paper copy in front of you so you know what goes where. And you have to check every space and every punctuation mark as well to make sure you have it correct.

My suggestion is to contact the shop you bought it from and try to get your money back. It's a major botch job.

Considering Nyssa has already fixed it, despite your pronouncement, that claim rings a little hollow...

[BetterRed]

Originally Posted by eschwartz

Considering Nyssa has already fixed it, despite your pronouncement, that claim rings a little hollow...

Jon - People who say it cannot be done should not interrupt those who are doing have already done it. George Bernard Shaw

BR

[JSWolf]

Originally Posted by BetterRed

Jon - People who say it cannot be done should not interrupt those who are doing have already done it. George Bernard Shaw

BR

But there is no guarantee that it's done correctly. There's no guarantee that all the lines are in all of the correct paragraphs. No matter how much is looks OK, it very well might not be OK.

[eschwartz]

Originally Posted by JSWolf

But there is no guarantee that it's done correctly. There's no guarantee that all the lines are in all of the correct paragraphs. No matter how much is looks OK, it very well might not be OK.

1. That is not what you said at first.
2. There is a possible cause for concern, in that the paragraph breaks will be in the wrong place (based on my previous experience doing much the same thing, one time). However, that will become evident during the reading, if it is indeed a problem, which I doubt.
   As long as it does not interrupt the reading flow, and the book still reads the same, I would consider it a waste of effort* to put in the extra work for almost zero return.

* -- Being a perfectionist, I did my next reread with the book and working version of the ebook side-by-side and fixed it anyway.