Handling Broken Paragraphs

[crutledge]

For whatever reason, in some PG files some paragraphs are broken into pieces. Some are one line paragraphs.

This requires a tedious process of code editing to put them back together.

Is there a method, which I have not found, of marking contiguous paragraphs and pulling them together into a single paragraph?

This is a lot to asks for, but it would sure come in handy.

[theducks]

Originally Posted by crutledge

For whatever reason, in some PG files some paragraphs are broken into pieces. Some are one line paragraphs.

This requires a tedious process of code editing to put them back together.

Is there a method, which I have not found, of marking contiguous paragraphs and pulling them together into a single paragraph?

This is a lot to asks for, but it would sure come in handy.

If they are TXT files, I use Notepad++ "unwrap text" to help.
You highlight the Paragraph and select un-wrap text (hint: assign a key-stroke, I use the + on the number pad as a simple stroke)

It is still tedious.
I expected better from PG.
Once they are converted from TXT, this is no longer an option.

[pietvo]

You can do this with the replace function (in the Edit menu).
Suppose you have the following text:

This is a part of

a broken paragraph.

These are really.

Two paragraphs.

You can mark the paragraphs to be merged by putting some text at the merge points that doesn't occur anywhere in the document. This could be something like XXXXX or as I have chosen: a single strange character like §. Now go to the code view. It looks like:

  <p>This is a part of§</p>

  <p>a broken paragraph.</p>

  <p>These are really.</p>

  <p>Two paragraphs.</p>

Now copy the part from the § up to and including the <p> on the next line. Put the cursor before the first text to be merged. Then choose the Replace function from the edit menu, and paste the copied text in the 'Find what' field. It will look like

§</p>    <p>

but in reality one or more of the spaces will be a newline character. In the 'Replace with' field enter a single space character. Then hit Replace all.

This supposes that all the broken paragraphs are the same, i.e. the same amount of newlines and spaces in between. If this is not the case you have to use a regular expression. Click the 'More' button and select Regular Expression, and in the 'Find what' field use:

§</p>\s*<p>

In a regular expression \s* means zero or more whitespace.
Sometimes there may be additional whitespace at the end of the first part or the beginning of the second part. To get rid of these add additional \s* at the proper places. E.g.

\s*§\s*</p>\s*<p>\s*

There may also be extra linebreaks between the parts, which manifests themselves in the code view as <br /> or something similar. To get rid of these you add (<br\s*/>)? at the proper place. This means zero or one line breaks. And the / in the line break may be preceded by zero or more whitespace. So the total 'Find what' field then could become a bit more complicated:

\s*§\s*</p>\s*(<br\s*/>)?\s*<p>\s*

[capidamonte]

I assume you use Windows.

There is a great old tool for unmangling text, called InterParse. It's available here on the forums.

InterParse Thread.

Bit of a funky interface, but once you get it you can do a LOT of different regex and search-replace without having to know any code.

cap

[crutledge]

\s*§\s*</p>\s*(<br\s*/>)?\s*<p>\s*

Pietvo, you're a genius. I hadn't thought of using a special marker character. Now I know the solution when I run into it again.

Many thanks!

[Pablo]

This is the feature I miss most in Sigil, and the main reason why I stick to BookDesigner.
In BookDesigner you just select "Broken sentences" in the "Element browser", and all the broken paragraphs show up. You click on each one and go straight to where the problem is.

[pietvo]

If the broken paragraphs can be characterized as not ending with a . ! or ? then you don't even have to add a special marker character but you can replace the § in the regular expressions with: ([^.!? ]) and put in the 'Replace with' field: \1 followed by a space character. Note: there is a space character after the ?

Now all the broken paragraphs will be found automatically.

[Jellby]

Paragraphs can also end in a :, and one should be especially careful with poetry lines.

[crutledge]

This has been a very interesting discussion. Any general method of addressing broken paragraphs can be fraught with problems.

I've used BD for hundreds of PG books and never found the Broken Paragraph selection to be particullaly useful. One advantage that BD does have is the HTML Fragment. A block of text can be highlighted and only the code for that fragment is presented for operations and repair. This eliminates the danger of affecting code throught the document.

If Valoric is following this thread, perhaps he would comment on the difficulty of implementing such a capability before anyone (me?) submits an issue. It may already be in the pipeline.

Right now I'm still partial to the special character due to the Law of Unintended Consequences of RegEx's. There is no cut and try methodology as with RegEx Buddy and other RegEx tools. Right now the programmer's best friend (UNDO) is problematical.

Thanks for all of the thought provocing discussion.

[Valloric]

Originally Posted by crutledge

One advantage that BD does have is the HTML Fragment. A block of text can be highlighted and only the code for that fragment is presented for operations and repair. This eliminates the danger of affecting code throught the document.

If Valoric is following this thread, perhaps he would comment on the difficulty of implementing such a capability before anyone (me?) submits an issue. It may already be in the pipeline.

Frankly I don't see this "edit HTML fragment" feature as particularly useful in Sigil. It's useful in BD where there is no Code View and you can't sync the position of the cursor in the rendered view to the HTML code view window.

This idea that it "eliminates the danger of affecting code throught the document"... I don't see it. You could certainly introduce problems in a fragment that could adversely affect the rest of the document.

But if you're talking about incorrect regexes sometimes affecting more than they should, that's different. A "Current selection" option for the "Look in" field in the search & replace dialog should alleviate that, and this feature is planned. Using that would make the S&R dialog operate only within the selected text.

[crutledge]

Originally Posted by Valloric

But if you're talking about incorrect regexes sometimes affecting more than they should, that's different. A "Current selection" option for the "Look in" field in the search & replace dialog should alleviate that, and this feature is planned. Using that would make the S&R dialog operate only within the selected text.

Same effect. Outstanding solution.

[DoctorOhh]

Originally Posted by crutledge

For whatever reason, in some PG files some paragraphs are broken into pieces. Some are one line paragraphs.

I know most of you are far cleverer then I am so this might help those folks who are my peers in limited ability.

I find the below scenario often enough after converting to epub from X format in calibre. I find Sigil invaluable in correcting these errors. It usually takes me a few find and replaces to get the book with broken paragraphs close to normal.

My most recent case went like this. All paragraphs were surrounded by <p class="MsoPlainText">paragraph words</p> and spaces between the paragraphs were created by <p class="MsoPlainText"></p>

1. I just replaced all <p class="MsoPlainText"></p> with <br/><br/>

2. Then I replaced(removed) all <p class="MsoPlainText"> with nothing and let Sigil clean up the closing elements.

3. Then I replaced all <br/> with </p><p> (again Sigil cleaned up the cases where there were <p></p> without words in between.)

4. Then I replaced all <p> with <p class="description"> which is the class that held my indent and margins.

After 4 steps all paragraphs were reformed and the book although not up to production quality specs was readable with paragraph indents and the spacing between paragraphs that I prefer.

Any lists that existed prior to this process required manual editing to make them whole again. It is probably best to identify lists upfront and change them so they are not caught up in this process.

[deckoff]

OK, so I go with " [^.]</p><p class="calibre2"> " Everything goes fine for me, but when I press replace(which is empty) the last symbol of the word is deleted-
For example

the same</p><p class="calibre2">

=

the sam

What to put in replace field to keep my last symbol.
help, pls..

[capidamonte]

You're selecting the last character before the close paragraph tag with [^.] so when you replace you have to put it back, otherwise it gets overwritten with your replace string (nothing, in this case).

Search: ([^.])</p><p class="calibre2">
Replace: $1

$1 can also be \1 on some regex systems. The enclosing paragraphs mark the match for reuse in replacements. Every successive set of paragraph tags (up to 9, I think) can be returned to your replacement string via $1, $2, etc.

cap

[Valloric]

Originally Posted by capidamonte

$1 can also be \1 on some regex systems. The enclosing paragraphs mark the match for reuse in replacements. Every successive set of paragraph tags (up to 9, I think) can be returned to your replacement string via $1, $2, etc.

Sigil uses \1, \2, \3... etc.