Removing Everything But Formatted Text

[Dybbuk]

I've been looking for a way on Sigil to delete everything in an epub but the stuff between <p> tags. In other words, to remove everything in a file but <p.*/p>.

It's easy to remove all the non p-tags with a regex - and wind up with plain text - but I'm stumped about how to remove all the non p-tags except the ones within paragraphs (such as <span>, <em>, etc.). I've Googled around and the consensus seems to be that regex is useless for parsing nested HTML tags. Is that really true?

[theducks]

Originally Posted by Dybbuk

I've been looking for a way on Sigil to delete everything in an epub but the stuff between <p> tags. In other words, to remove everything in a file but <p.*/p>.

It's easy to remove all the non p-tags with a regex - and wind up with plain text - but I'm stumped about how to remove all the non p-tags except the ones within paragraphs (such as <span>, <em>, etc.). I've Googled around and the consensus seems to be that regex is useless for parsing nested HTML tags. Is that really true?

Why not use Calibre to convert to TXT?
The Convert the (cleaned of all style) TXT to HTML/EPUB first.

IIRC Notepad++ can strip HTML

[DiapDealer]

I've Googled around and the consensus seems to be that regex is useless for parsing nested HTML tags. Is that really true?

Maybe not utterly useless, but it's certainly not easy or foolproof using regex. To do it right, you need something capable of actually parsing html (which regex is certainly incapable of doing).

[Dybbuk]

Originally Posted by theducks

Why not use Calibre to convert to TXT?
The Convert the (cleaned of all style) TXT to HTML/EPUB first.

IIRC Notepad++ can strip HTML

I don't want to strip the style. I want to keep the style and formatting, and remove everything outside of the paragraphs, such as <div>, <script>, etc.

I'd prefer to use Sigil, but how would I do this on Notepad ++?

[Dybbuk]

Originally Posted by DiapDealer

Maybe not utterly useless, but it's certainly not easy or foolproof using regex. To do it right, you need something capable of actually parsing html (which regex is certainly incapable of doing).

In that case, why does Sigil - whose whole purpose is editing HTML - use regex? (Not complaining, Sigil is awesome. Just curious.)

[DiapDealer]

Originally Posted by Dybbuk

In that case, why does Sigil - whose whole purpose is editing HTML - use regex? (Not complaining, Sigil is awesome. Just curious.)

Because there's a difference between editing and parsing. And the Find & Replace feature in most editing software really has nothing to do with parsing code. F&R doesn't even know what "code" is. It's simply searching text for patterns you specify. Regex just happens to be one of the most flexible/powerful and common ways to achieve this.

What you want to do goes beyond the normal definition of editing or even Searching & Replacing. You're looking for something that automatically transforms code into new code—new code whose conventions you want to be able to specify (and preferably with no data-loss). That's a whole different ball o' wax.. and not something that's easily incorporated into a program (not without hard-coding the transformation rules, anyway; which would seriously limit the feature's usefulness to an end-user).

I don't want to strip the style. I want to keep the style and formatting, and remove everything outside of the paragraphs, such as <div>, <script>, etc

What about headers? Blockquotes? There's all kinds of situations that can arise in ePubs where text you definitely don't want to lose occurs outside of the <p> tags.

[Dybbuk]

Originally Posted by DiapDealer

Because there's a difference between editing and parsing. And the Find & Replace feature in most editing software really has nothing to do with parsing code.

Okay, but we're talking about Sigil here. It has everything to do with parsing code. If regex can't parse HTML, why does Sigil use regex?

Originally Posted by DiapDealer

F&R doesn't even know what "code" is. It's simply searching text for patterns you specify. Regex just happens to be one of the most flexible/powerful and common ways to achieve this.

I'll believe that regex is flexible and powerful when it allows me to differentiate between stuff within a paragraph and stuff outside a paragraph. This is incredibly basic parsing, and I'm shocked that regex can't handle it. Why can regex easily find "<p.*/p>" but is totally incapable of identifying stuff that is NOT "<p.*/p>."?

Originally Posted by DiapDealer

What you want to do goes beyond the normal definition of editing or even Searching & Replacing. You're looking for something that automatically transforms code into new code—new code whose conventions you want to be able to specify (and preferably with no data-loss). That's a whole different ball o' wax.. and not something that's easily incorporated into a program (not without hard-coding the transformation rules, anyway; which would seriously limit the feature's usefulness to an end-user).

Huh? All I want to do is delete stuff that's not between <p> and </p>! Sigil can easily find stuff that IS between <p> and </p>. It should be just as easy to select stuff that is NOT between them. It's text selection, nothing more. Trust me, I'm not attempting to sabotage or revolutionize code conventions. I'm just a humble epub converter trying to make sense of modern software.

What about headers? Blockquotes? There's all kinds of situations that can arise in ePubs where text you definitely don't want to lose occurs outside of the <p> tags.

I definitely want to lose all of the stuff outside of the p tags. P-tags aren't really the issue. I just want to know why I can select things between a certain parameter but not things outside the same parameter.

[DiapDealer]

Sorry, I tried as best I could to explain, but apparently failed. Unfortunately, you may have to accept the fact that Sigil can't do what you want without understanding why.

[theducks]

(?sm)</p>\s+(.+?)\s+<p>
Should work to remove things outside those tags
Don't try this on any copy you want to be usable after you are done,

BUT YOU WERE WARNED that there are other valid things between the closing </p> and the Next <p> that should not be removed: The list is big, so I am not wasting my time typing it.

[exaltedwombat]

Originally Posted by Dybbuk

I don't want to strip the style. I want to keep the style and formatting, and remove everything outside of the paragraphs, such as <div>, <script>, etc.

I'd prefer to use Sigil, but how would I do this on Notepad ++?

What makes you think all style and formatting is defined inside the <p> tags?

[Dybbuk]

Originally Posted by DiapDealer

Sorry, I tried as best I could to explain, but apparently failed. Unfortunately, you may have to accept the fact that Sigil can't do what you want without understanding why.

Very true! Sorry If I came across as pissy, I just don't understand why Sigil can choose expression x but not everything besides expression x. It can choose the letter "Q" and every letter besides "Q".

[Dybbuk]

Originally Posted by exaltedwombat

What makes you think all style and formatting is defined inside the <p> tags?

I don't. I was just trying to make clear that I don't want to erase the style and formatting tags inside paragraphs.

[Dybbuk]

Originally Posted by theducks

(?sm)</p>\s+(.+?)\s+<p>
Should work to remove things outside those tags
Don't try this on any copy you want to be usable after you are done,

BUT YOU WERE WARNED that there are other valid things between the closing </p> and the Next <p> that should not be removed: The list is big, so I am not wasting my time typing it.

Cool! I've tried it on several epubs but it often selects p-tag stuff. Maybe I'm doing it wrong? I'm using Sigil 7, under various search settings. Earlier I tried exploiting the fact that angle brackets within paragraphs usually have a space before them with this regex:

\s<[^p/]

Which works, but only occasionally. It operates under the assumption that the epub is well-formed, with lines between non-p tags and spaces between other tags.

You're my hero! I'll definitely tinker with this regex some more!

[mrmikel]

With your fiddling with regex, you can make whole sections of text disappear in such a way that you won't see them until you have invested some amount of time afterwards in further editing.

With this kind of experimentation, you want to save with a new name every five minutes at least.

If there is someone else who is familiar with your material, it would be beneficial for them to double check you because after fiddling for an hour or so, you will not see such changes or maybe even care!

[exaltedwombat]

Are we really asking to strip everything except paragraphs, bold and italic? I suppose it's a fair assumption these might always be inline, inside the <p> tags. Might be a good idea to retain header tags as well?