Oddity in Mend and Prettify

[KevinH]

I am confused. That was the point of the requested change for "Mend and Prettify", ie. that it check for well-formed before running it. It forces people to not run that command until after the source is well-formed. So just Mend or hand fixing until well formed, then running Prettify.

"Mend and Prettify" is actually not two different steps. It parses any xhtml using gumbo (whatwg) and the when serializing the DOM tree back to xhtml, it does the prettify part.

The actual name of the command indicates how it works (ie it runs the gumbo repair parser then when serializing it back it does the prettify part. Normally that is safe but as you saw it can cause issues when attributes are not properly quoted, and since Prettify changes whitespace wholesale, it can be a bit disruptive at times.

So Mend vs Mend and Prettify are two different commands that each first parse the xhtml using a repair/whatwg parser then just serialize the DOM tree in different ways. The latter has never been running Mend first then Prettify.

[DNSB]

I may have misunderstood but I was under the impression that Mend and Prettify ran mend and then prettify. By the evidence, Mend and Prettify does not run Mend and then Prettify and so it is useless to me. I ended up renaming the option to Prettify only since that change helps to remind me to run it only if the ePub does not need mending. I would suggest removing the Mend portion of the option and having only Prettify as the label. I also note that Mend & Prettify still has the P underlined and Mend has the M underlined as though there were keyboard shortcuts for them.

[KevinH]

Mend and Prettify effectively still does a "mend" (ie parses using the whatwg gumbo parser) but if well-formed first the parser makes no changes, so just the serialization of the parsed DOM tree will make changes.

I am happy to rename Mend and Prettify with just Prettify, but the parsing step will still need to be done by gumbo. It is inherent in how Prettify is done.

[DNSB]

Since Mend & Prettify fails on a missing DOCTYPE in one file and makes no changes to any files, I really don't consider it as doing a mend. If I click on the Mend, the missing DOCTYPE is inserted and then Prettify (AKA Mend & Prettify) will work to prettify the ePub.

Basically to me, if Mend & Prettify will only work on a ePub that does not need mending, it is not Mend & Prettify, it is just Prettify with a longer name.

[KevinH]

Which was the exact feature requested in this thread. So I can either revert that change and leave Sigil as it has worked for years, or keep that change and rename that menu item to just Prettify.

Which do people want?

[DNSB]

Since my original complaint was about an single ePub where Mend & Prettify mucked up the file, I see the point in no longer allowing Mend & Prettify to run on a file with errors. OTOH, since running any of epubcheck/epubveri/well-formed showed those errors as did opening in Preview, I've changed my workflow to ensure an error checker is run before doing a Mend & Prettify which I must admit was mostly used to automatically fix the missing DOCTYPEs.

I ran well-formed check, epubcheck and epubveri on a ePub3 file with a missing DOCTYPE and none showed an error.

Since the alternative of running Mend will cause the same issue as running Mend & Prettify, I would vote for reverting the change with perhaps a mention in the User Guide about potential issues though it already mentions checkpointing before/after though that seems more about the white space changes.

[BeckyEbook]

There must be more to it… After all, the DOCTYPE is just a minor detail; there must be something worse going on.

And what does the "Well-Formed Check EPUB" (F7) say?

Please try another test:

• Take your already-processed file, with the DOCTYPE and everything else perfect.
• Run "Mend and Prettify."
• Does it work? OK, let's move on.
• In one of the XHTML files, remove the DOCTYPE.
• Run "Mend and Prettify."
• Does it work? It does for me.

---

I don’t want to make life difficult for anyone or change the habits users have developed over the years while working with Sigil.
For me, this extra safeguard is useful, and even if it’s removed, I can still add it to my build.

[DNSB]

I'll give it a try.

• Opened a known good ePub. Well-formed, epubcheck and epubveri showed no issues.
• Ran Mend and Prettify. Pretty Printing windows popped up and ran.
• Removed DOCTYPE from 1 file. Mend and Prettify popped up the error window listing that file. See attached image.
• Removed DOCTYPE from 2 more files. The error window mentioned the first file. The popup was identical.
• Ran Mend. The 3 missing DOCTYPEs were restore and M&P ran.

Well-formed check, epubcheck and epubveri did not show any errors for the missing DOCTYPE which is correct since ePub3 does not require the DOCTYPE. Ghod alone knows where Sigil is picking it up as an error.

Hmmm... latest code from GitHub, Python 3.14.6, Qt 6.10.2, Microsoft (R) C/C++ Optimizing Compiler Version 19.51.36252 for x64

[BeckyEbook]

You're right! I've modified this commit so much that it works differently for me than the official version from the master branch.

I hope I can come up with changes that will satisfy everyone, because the current version is way too strict about ensuring the files are correct before Prettify.

[KevinH]

Well having a doctype was required for epub2 to allow for named entities. For epub3 the doctype is still required for the spec case and I believe that epubcheck has a long standing bug about not requiring it. Only html allows it to be skipped. Sigil will always treat missing doctype as a error for both epub2 and epub3. Sigil also requires the xml header as well. Neither of those are changing. Calibre simply cares less about doctypes and epubcheck and spec meeting than Sigil, as that has always been Kovids view and why calibre has its own check function.

I am travelling now but upon my return I will simply revert this change, and we will go back to how it has worked for years. Sigil will continue to require the doctype as the original epub3 spec clearly required it for xhtml no matter what epubcheck does. Only html said it was optional.

[BeckyEbook]

Wait! Take a look at this solution.
I hope it will suit everyone involved. If not – OK, just pull your commit entirely.

If we don’t use the main FileIsWellFormed but instead use the existing GumboWellFormedErrorForSource, then IMHO we get exactly what we expected:
* important issue like the DOCTYPE gets fixed either way
* but if there’s some strange markup or a problem like the one in the first post – we see the message

@DNSB: Check for yourself if it’s really that bad.

[KevinH]

I will not be able to examine or test this for a few weeks. But upon my return if you both agree and if it works as expected, and both think this is better than reverting, I will include it along with a name change to just Prettify.

[rkomar]

Maybe rename it to "Mend if possible, and if so, Prettify"