0.7.700 still has bug...

[crutledge]

Originally Posted by PeterT

Look for the thread that tells you that there is a version available for a few platforms for Alpha testing.

I don't find anything.

[KevinH]

Originally Posted by crutledge

I don't find anything.

https://www.mobileread.com/forums/sho...25&postcount=1 has the download link

And the two associated threads are here:

https://www.mobileread.com/forums/sho...d.php?t=246400
https://www.mobileread.com/forums/sho...d.php?t=246005

[JimmyG]

Used preview on a file and it told me there was an error just before the </body> tag (where most Sigil errors occur,) had to upload the file to W3C to find it. Missing </p>, nowhere near the </body>. As I said, not that helpful.

I've discovered that the p button won't work until I've closed the enclosing blockquote tag. Even the smarty quotes plugin does error checking. That's just obsessive.

Lurking somewhere in all that error checking is a routine that tries to fix the error without asking the user first, and that is what is truncating files. And that is a BUG. To destroy hours, days, or even weeks of work without notice either before or after is simply inexcusable, and I don't care what anyone says about structure, design, blah, blah, blah.

Hopefully, I can finish this one book, and then Sigil goes back on the shelf. Perhaps some can write error-free html on the first pass, but I can't. I leave the error checking to the end (EpubCheck on idpf) just before and after the final proofread.

[eschwartz]

At the risk of sounding slightly repetitive... is there any particular advantage (for your use) of Sigil over calibre's Edit Book?

The calibre editor does not change your code without your express permission (by using either Beautify or Fix HTML), and you can run Check Book to list errors for manual fixing. It is less invasive, and much more geared towards people who are editing the raw XHTML.

[KevinH]

Hi,
Again, please provide a repeatable set of steps and a sample file, and I would be happy to look into this and try to fix it if it is really eating text. And I agree ... No matter how poor the tags are, text node information should never be lost. So if it is losing text, I too consider that a bug.

That said, I have never lost any text using Sigil. So I need some help tracking down where and how this is happening. That is why I need a sample file and sequence of steps to find the true cause of your issue.

I do agree that sometimes Sigil description of where to look for the issue leaves a lot to be desired. Finding and fixing single missing tags could be done in a plugin if we find out the issue is in Tidy someplace.

So if you can recreate any of your "eating text" bugs repeatable I would be happy to try and fix it.

KevinH

Originally Posted by JimmyG

Used preview on a file and it told me there was an error just before the </body> tag (where most Sigil errors occur,) had to upload the file to W3C to find it. Missing </p>, nowhere near the </body>. As I said, not that helpful.

I've discovered that the p button won't work until I've closed the enclosing blockquote tag. Even the smarty quotes plugin does error checking. That's just obsessive.

Lurking somewhere in all that error checking is a routine that tries to fix the error without asking the user first, and that is what is truncating files. And that is a BUG. To destroy hours, days, or even weeks of work without notice either before or after is simply inexcusable, and I don't care what anyone says about structure, design, blah, blah, blah.

Hopefully, I can finish this one book, and then Sigil goes back on the shelf. Perhaps some can write error-free html on the first pass, but I can't. I leave the error checking to the end (EpubCheck on idpf) just before and after the final proofread.

[DiapDealer]

Originally Posted by JimmyG

Even the smarty quotes plugin does error checking. That's just obsessive

As the creator/maintainer of the plugin in question, I can assure you it does absolutely NO html/css/epub/structure error-checking. It doesn't really give the slightest damn if your code is fubar or not.

I agree with KevinH. Lets get some repeatable steps to nail down under what conditions things get eaten. That's the only way things will ever be fixed. Putting Sigil on the shelf won't help at all.

[JimmyG]

Originally Posted by DiapDealer

As the creator/maintainer of the plugin in question, I can assure you it does absolutely NO html/css/epub/structure error-checking. It doesn't really give the slightest damn if your code is fubar or not.

I agree with KevinH. Lets get some repeatable steps to nail down under what conditions things get eaten. That's the only way things will ever be fixed. Putting Sigil on the shelf won't help at all.

I tried to use your plugin on a file, and it refused to do it because the file had an error or was not well-formed, I forget the actual wording. Found the error, fixed it, and then no problem. That's all I know.

I don't know how plugins work, but perhaps Sigil checks the file before it hands it over to your plugin?

[JimmyG]

Originally Posted by KevinH

Hi,
Again, please provide a repeatable set of steps and a sample file, and I would be happy to look into this and try to fix it if it is really eating text. And I agree ... No matter how poor the tags are, text node information should never be lost. So if it is losing text, I too consider that a bug.

That said, I have never lost any text using Sigil. So I need some help tracking down where and how this is happening. That is why I need a sample file and sequence of steps to find the true cause of your issue.

I do agree that sometimes Sigil description of where to look for the issue leaves a lot to be desired. Finding and fixing single missing tags could be done in a plugin if we find out the issue is in Tidy someplace.

So if you can recreate any of your "eating text" bugs repeatable I would be happy to try and fix it.

KevinH

It's a stealth bug. It's never done it right in front of me. Like that test file. I did everything I could think of and the file remained pristine. Went back to work on another file, finished it, went back and checked the test file, and found the last half deleted.

I'm following everyone's suggestions: staying away from book view, using preview (I didn't even know it was there,) not saving until I've checked that everything is error free, etc. But I'm still saving a copy of each file, because each requires considerable work that I don't want to have to repeat if it disappears.

I have a fallback position. When I gave up on 7.4 I was working on a big book that I had been working on for months. I took a time out and wrote a program using the MS Net framework just to finish that book. I'm 89% through the final proofread. When I get it done, I'll go back and make the program more general purpose so I can make new books with it. It doesn't eat files, there's no error-checking, but syntax highlighting and spell-checking are irritatingly sloooow.

[JimmyG]

Originally Posted by eschwartz

At the risk of sounding slightly repetitive... is there any particular advantage (for your use) of Sigil over calibre's Edit Book?

The calibre editor does not change your code without your express permission (by using either Beautify or Fix HTML), and you can run Check Book to list errors for manual fixing. It is less invasive, and much more geared towards people who are editing the raw XHTML.

I don't like Calibre.

[PeterT]

Originally Posted by JimmyG

I don't like Calibre.

But you can use the editor without adding the book to a library or doing any other processing with calibre.

It's foolish to cut off your nose to spite your face.

[DiapDealer]

Originally Posted by JimmyG

I tried to use your plugin on a file, and it refused to do it refused to do it because the file had an error or was not well-formed, I forget the actual wording. Found the error, fixed it, and then no problem. That's all I know.

The underlying SmartyPants algorithm needs to tokenize the html so that it can isolate the text of the document (otherwise it would be "smartening" the quotes around the tags various attributes). Of course it will barf it is fed garbage. The only other possible thing it could do is try to fix that garbage (which I have no desire to do, and you've clearly indicated you don't want programs doing).

I understand not wanting to have your code altered without your permission--I really do. But I don't quite understand what you expect an automated markup-manipulation tool (which relies on well-formed markup to be able to function properly) to do when it encounters problems with your markup? It has exactly two choices: barf, or try to fix the problem.

Even calibre does this. If you want to Beautify or Smarten or Fix your html, it will make the changes it deems necessary. And the use of any of those automated tools requires ancillary changes to the markup that are a bit unrelated to what the Tool is trying to do. For instance, the built-in calibre Smartener requires that you have to sacrifice any HTML entities. I don't say this to condemn calibre--but just to point out the similarities. Calibre's checkpoint system and visual compare make it easy to revert any undesired changes, and I love that. I use it a lot. But I also know that I can close Sigil without saving if I suspect anything weird might have happened. I do that a lot, too.

My long, and rather drawn out, point is this: the time to use any automated markup-manipulation tools is AFTER you've determined you have clean markup.

[KevinH]

Hi,

I have been somewhat able to duplicate the "eating text" issue but I had to do something really horrible to get Tidy to actually "eat text". I had to confuse it as to what is text and what is tag.

<?xml version="1.0" encoding="UTF-8" standalone="no" ?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"><html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title/>
<link href="../Styles/Style0001.css" rel="stylesheet" type="text/css"/></head>
<body>
<h1>Test</h1>

<p>Now is the time for all good men to come to the aid of the party 1.</p>

<p>Now is the time for all good men to come to the aid of the party 2.</p>

<pclass="bot" Now is the time for all good men to come to the aid of the party 3.</p>

<p>Now is the time for all good men to come to the aid of the party 4.</p>

</body>
</html>

Notice the missing ">" to mark the end of the bad <pclass="bot" tag. Unmatched or missing ">" and "<" will always drive cleaning programs insane as it can't tell what is text and what is tag.

Here is what Tidy in Sigil did to this file on open:

<?xml version="1.0" encoding="UTF-8" standalone="no" ?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"><html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title/>
<link href="../Styles/Style0001.css" rel="stylesheet" type="text/css"/></head>
<body>
<h1>Test</h1>

<p>Now is the time for all good men to come to the aid of the party 1.</p>

<p>Now is the time for all good men to come to the aid of the party 2.</p>

<p>Now is the time for all good men to come to the aid of the party 4.</p>

</body>
</html>

A better solution when unpaired "<" ">" exists is to simply replace the unbalanced one with its html entity codes

 "<"  or ">"

to prevent anything from being lost.

The problem is Tidy is really a mess to try and fix or change. So I think the only solution in the long run is to write a much simpler replacement for Tidy, that defaults to creating too much "text" as opposed to deleting any.

If I get a free moment, I may take a stab at a prototype for doing this in python to create a sort of "safe clean" plugin to see if it actually is doable and would help.

Parsing bad xhtml especially with unmatched "<" and ">" is fraught with issues as it can confuse the hell out of the parser.

A "safe clean" parser would create the following output for that example:

&lt;pclass="bot" Now is the time for all good men to come to the aid of the party 3.<p></p>

Not great but still easier to fix after the fact with no text lost.

Would this be a help?

KevinH

[JimmyG]

Originally Posted by KevinH

Hi,

I have been somewhat able to duplicate the "eating text" issue but I had to do something really horrible to get Tidy to actually "eat text". I had to confuse it as to what is text and what is tag.

<?xml version="1.0" encoding="UTF-8" standalone="no" ?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"><html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title/>
<link href="../Styles/Style0001.css" rel="stylesheet" type="text/css"/></head>
<body>
<h1>Test</h1>

<p>Now is the time for all good men to come to the aid of the party 1.</p>

<p>Now is the time for all good men to come to the aid of the party 2.</p>

<pclass="bot" Now is the time for all good men to come to the aid of the party 3.</p>

<p>Now is the time for all good men to come to the aid of the party 4.</p>

</body>
</html>

Notice the missing ">" to mark the end of the bad <pclass="bot" tag. Unmatched or missing ">" and "<" will always drive cleaning programs insane as it can't tell what is text and what is tag.

Here is what Tidy in Sigil did to this file on open:

<?xml version="1.0" encoding="UTF-8" standalone="no" ?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"><html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title/>
<link href="../Styles/Style0001.css" rel="stylesheet" type="text/css"/></head>
<body>
<h1>Test</h1>

<p>Now is the time for all good men to come to the aid of the party 1.</p>

<p>Now is the time for all good men to come to the aid of the party 2.</p>

<p>Now is the time for all good men to come to the aid of the party 4.</p>

</body>
</html>

A better solution when unpaired "<" ">" exists is to simply replace the unbalanced one with its html entity codes

 "<"  or ">"

to prevent anything from being lost.

The problem is Tidy is really a mess to try and fix or change. So I think the only solution in the long run is to write a much simpler replacement for Tidy, that defaults to creating too much "text" as opposed to deleting any.

If I get a free moment, I may take a stab at a prototype for doing this in python to create a sort of "safe clean" plugin to see if it actually is doable and would help.

Parsing bad xhtml especially with unmatched "<" and ">" is fraught with issues as it can confuse the hell out of the parser.

A "safe clean" parser would create the following output for that example:

&lt;pclass="bot" Now is the time for all good men to come to the aid of the party 3.<p></p>

Not great but still easier to fix after the fact with no text lost.

Would this be a help?

KevinH

Does Tidy come into play, even tho I have it turned off?

Your example does not show the problem I have found. Every file I have ever lost, in 7.4 and 7.7 does the same thing. From some apparently arbitrary point (perhaps a mistake?) in the file, it removes everything from that point up to </body> and rewrites the ending to </body></html> without the line break.

[JimmyG]

Originally Posted by DiapDealer

The underlying SmartyPants algorithm needs to tokenize the html so that it can isolate the text of the document (otherwise it would be "smartening" the quotes around the tags various attributes). Of course it will barf it is fed garbage. The only other possible thing it could do is try to fix that garbage (which I have no desire to do, and you've clearly indicated you don't want programs doing).

I understand not wanting to have your code altered without your permission--I really do. But I don't quite understand what you expect an automated markup-manipulation tool (which relies on well-formed markup to be able to function properly) to do when it encounters problems with your markup? It has exactly two choices: barf, or try to fix the problem.

Even calibre does this. If you want to Beautify or Smarten or Fix your html, it will make the changes it deems necessary. And the use of any of those automated tools requires ancillary changes to the markup that are a bit unrelated to what the Tool is trying to do. For instance, the built-in calibre Smartener requires that you have to sacrifice any HTML entities. I don't say this to condemn calibre--but just to point out the similarities. Calibre's checkpoint system and visual compare make it easy to revert any undesired changes, and I love that. I use it a lot. But I also know that I can close Sigil without saving if I suspect anything weird might have happened. I do that a lot, too.

My long, and rather drawn out, point is this: the time to use any automated markup-manipulation tools is AFTER you've determined you have clean markup.

Faced with a missing < or >, my program blithely smartens the quotes around attribute values. Of course, that situation would also screw up my syntax highlighting which would make the problem immediately apparent. Since my program is only for my use, it smartens on file load because I always use curly quotes. It also replaces quotes as I type. I have an equivalent for preview. All it does is show the file as it would look with the html and css in the file. If something is wrong, a missing < or >, it will be obvious.

I realize producing something for the world to use is a lot different from my producing something that only I use.

Sigil kinda reminds me of a grammar school teacher I had, whom we all loved so much. If you had a missing comma in your paper, she would take her red pencil and cross out the whole paragraph.

[JimmyG]

Originally Posted by PeterT

But you can use the editor without adding the book to a library or doing any other processing with calibre.

It's foolish to cut off your nose to spite your face.

Went and took another look and saw this "...includes a comprehensive checking tool that can find many of the most common errors in book structure and formatting and even fix some of them automatically."

That bothers me, but I may give it a shot. The editor looks a lot different from the last time I looked at it.