buggy behaviour generating htmlToC

[philja]

I'm getting strange behaviour when trying to generate a new html ToC in Sigil 2.8.0 on my MacBook (Tahoe 26.5.1 M2 chip). Both epub2 and epub3.

Tools > Table of Contents > Create HTML from Table of Contents: produces a warning dialog that there is an existing HTML Table of Contents - should Sigil overwrite?

The strange thing is that when over this dialog, the cursor changes to a small circle with alternate black and white quadrants. It is unable to click on the No or Yes buttons. A click elswhere in that box introduces a small cursor against its text as though the text had become editable, which it hasn't.

Even stranger, very occasionally an attempt to click produces a result. If the No box works, a new xhtml file is generated (sounds logical). If the Yes box works, the existing html ToC is overwritten.

But over at least 20 attempts, these results have only happened twice (once for No and once for Yes) so I can't provide a systematic way to use it.

I've tried to produce a screenshot of the cursor change but the cursor does not get included in the shot.

I haven't had an opportunity yet to try this on Windows or linux.

[philja]

A little more testing shows that a single click on the yes button with the strange cursor, followed by closing the dialog by clicking on the little red dot (Apple MacBook), gets a positive result.

The newly generated htmlTOC doesn't, however, align correctly in Code View with a click in Preview alongside the target. Code View aligns one line lower than desired. Before the regenration, it aligned correctly.

[BeckyEbook]

I can confirm that in Windows, the cursor also changes to "Busy," but this does not prevent you from selecting "Yes" or "No" in any way.

[philja]

I've now had a chance to try in Linux and Windows11 (both on Sigil 2.8.0) and the basic buggy behaviour is there but slightly different.

Linux: the cursor changes to a pair of self orbiting small circles (normally seen only during a 'Save' operation). Clicking on No produces a new htmlToC as expected. Clicking on Yes overwrites the existing with the new material as expected. Both cases automatically close the dialog (unlike in the MacBook).
Windows: same behaviour except the cursor changes to an egg timer shape.

In both Linux and Windows, after overwriting the htmlToC with the new mods, when clicking to the right of an entry in the Preview window, the Code View aligns one entry lower than desired (same as the MacBook).

Overall, something not quite right, but the Mac experience is worse.

Ho Ho, Becky - we were both writing at the same time!!

[BeckyEbook]

Originally Posted by philja

In both Linux and Windows, after overwriting the htmlToC with the new mods, when clicking to the right of an entry in the Preview window, the Code View aligns one entry lower than desired (same as the MacBook).

That's true, that does happen. However, I routinely use "Mend and Prettify," and after that it works fine. That's probably why I've never encountered this issue.

[KevinH]

If Preview is out of alignment, it is caused by one of the following:

- it contains illegal xhtml where Mend has to fix it (improperly nested tags, improper bare tags, etc)
- it is missing its doctype
- it is missing its xml header

All of the above are fixed by Mend (or Mend and Prettify).

That is called a Qt busy cursor. It is meant to be seen during any long operation. It should never prevent clicking on Yes or No in a message dialog. If so that is a Qt bug. If need be the busy cursor can be removed as it is only meant to provide user feedback.

I will look into it, but nothing has changed in this area of the code in a long time.

What exact version of Mac OS are you using?

FYI, MacOS 26 Tahoe is still not fully supported in Qt 6.10.2 due to its bugs and gui changes. MacOS really should stop its silly once a year new OS change cycle, and instead focus on stability improvements.

Update: On my Sequoia (15.7.5) machine, I can always 100% still select the button when the Qt busy cursor is present. So this must be a MacOS Tahoe thing?

[BeckyEbook]

I found it. Just add a new line before the DOCTYPE.

[DiapDealer]

Curious what might have changed that would make the new line necessary. Before the doctype seems like an odd place.

[KevinH]

No, it is missing its xml header line. This bug has existed since forever. Please add the missing xml header line before the doctype in TOCHtmlWriter (and remove the injected newline) in your PR and I will merge it.

Thanks!

And I will change that routine in MainWindow to remove the busy cursor immediately before the dialog box is shown and put it back immediately after.

[KevinH]

Ah! You were right! We were just missing a newline at the end of the xml header.
So we change the code that adds the xml header to include the newline and the proper charset!

Update

Of course the code is part of the QXmlStreamWriter class in Qt and it has no way of setting the charset or adding the new line to their StartDocument() routine!

[philja]

After making the new htmlToC file, the <?xml version....>and <!DOCTYPE....> are both written to the same line whereas they were not in the original htlmlToC file. The act of overwriting it caused that corruption.

Not a major problem because I can tell the others to run Mend & Prettify each time after the htmlToC regeneration.

I am using Tahoe 26.5.1 [original post] . I'm used to seeing over many years complaints about MacOS updates breaking all sorts of third-party applications. So it wouldn't surprise me if Apple were behind this one too.

But I was trying to explain to a Mac user that the simplest way to update the htmlToC after a user edit to some of the text files is to use Sigil's tool. It's a shame that the busy icon makes it look like Sigil has stopped working. It shouldn't be busy since neither the Yes nor the No button has been clicked. And when it is clicked (despite the busy icon's appearance) the dialog doesn't close on Mac (it does on Linux and Windows).

That gives a distinct impression on Mac that Sigil has stalled.

[KevinH]

So BeckyEbook, something like this might be a bit better:

    m_Writer = new QXmlStreamWriter(&out);
    m_Writer->writeStartDocument();
    m_Writer->writeCharacters("\n");
    if (version.startsWith('2')) {
        m_Writer->writeDTD("<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.1//EN\"\n"
                           "   \"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd\">\n");
        m_Writer->writeStartElement("html");
	m_Writer->writeAttribute("xmlns", "http://www.w3.org/1999/xhtml");
        m_Writer->writeCharacters("\n");
    } else {
        m_Writer->writeDTD("<!DOCTYPE html>\n");
	m_Writer->writeStartElement("html");
        m_Writer->writeAttribute("xmlns", "http://www.w3.org/1999/xhtml");
	m_Writer->writeAttribute("xmlns:epub", "http://www.idpf.org/2007/ops");
        m_Writer->writeCharacters("\n");
    }

Notice the extra m_Writer->writeCharacters("\n"); immediatly after the write to the startdocument.

[KevinH]

Originally Posted by philja

But I was trying to explain to a Mac user that the simplest way to update the htmlToC after a user edit to some of the text files is to use Sigil's tool. It's a shame that the busy icon makes it look like Sigil has stopped working. It shouldn't be busy since neither the Yes nor the No button has been clicked.

That gives a distinct impression on Mac that Sigil has stalled.

It works on all Mac OS versions before the broken Tahoe. Not sure why Tahoe changed that.

In reality, the code is busy searching the enitire book for existing HTML TOC, and performing many other checks before that prompt and after it. If you had a test epub with hundreds and hundreds of files, you would see the need for the busy icon.

But as I already said, I can drop the busy icon right before the message box and restore it, so that this works on Tahoe too. It has been that way for over 5 years with no issues on any of my Macs. So it is only a Tahoe thing.

Update:

The fix to workaround the Tahoe issue with using a busy cursor has been pushed to master.

[BeckyEbook]

Of course the code is part of the QXmlStreamWriter class in Qt and it has no way of setting the charset or adding the new line to their StartDocument() routine!

That's not entirely true.
WriteStartDocument is also used when creating NCX and encoding.xml files, but in those cases, QString isn't used – instead, QIODevice is used – and there's a setting from XMLWriter.cpp:

m_Writer->setAutoFormatting(true);
m_Writer->setAutoFormattingIndent(2);

So if we only have QString in the TOC, we need to add these new lines.
With QIODevice and AutoFormatting enabled, the encoding and newline characters are added automatically.

So I can add

m_Writer->writeCharacters("\n");

But the encoding still won’t be added (of course, Mend will fix this, for example, when saving).

Any method will do, as long as the DOCTYPE is on a new line.

[KevinH]

Please go with the m_Writer->writeCharacters(“\n”); added line and I will pull in the PR.

We either need to write the string to a new file or write that string to an existing file. It is easier kept as a string.