Sigil-0.9.13 Released

[KevinH]

One interesting test just before you go to bed at night would be to:

1. close the Preview window and exit from Sigil (to prevent the Preview Window from reopening)
2. restart Windows 10
3. start a fresh copy of Sigil and open the latest version of our User's Guide (so we are all starting with exactly the same epub). Verify that Preview is NOT open.
4. Then check and write down the memory usage of Sigil
5. Then close the lid to sleep for the night

Once you get up the next day.

6. Reopen the lid and **without** clicking on Sigil (which would start new gui threads) check the memory and write it down.

Let us know what if anything changed during sleep.

Then repeat the entire process the next night but with Preview ***Open***.

That way we can see how much memory consumptions changes overnight with and without Preview being open.

This may help isolate any increased memory usage to the Qt Webkit or not.

Thanks!

KevinH

[brent63]

Originally Posted by KevinH

What font or fonts are you using? Some fonts are broken when used with Qt. Not many but there are a few. Are you embedding fonts?

Yes I am embedding fonts.

Canterbury.ttf for dropcase.
Georgia.ttf for all other text.

Originally Posted by KevinH

Please open the Inspector widget at the bottom of Preview and open the html tree and verify that your "i" tag is being parsed properly. That tree shows exactly how your Code is being parsed and what css is active, etc.

I have opened widget at the bottom of the preview page, and gone thru the seven items there but don't see any issues?

Can you be more specific as to where to look?

Originally Posted by KevinH

What exact Windows 10 update are you using and what exact build number?

According to a Google researcher, there are slowdowns and memory leaks associated with a Windows 10 security feature called Control Flow Guard that actually never frees memory in processes that open a large number of threads with different starting locations (such as the chrome browser or Sigil).

This can be especially troublesome on machines with lower amounts of memory as the only way to get the memory and speed back is to restart Windows. Restarting the impacted program will not help.

The slowdown part has been fixed in a recent Windows build. According to Google researchers Microsoft still hasn't fixed the lack of memory being freed even when an app exits (ie worse than a leak which is freed when a program exits).

The newer Qt webkits use multiple threads to walk the javascript stack to garbage collect, multiple threads to load resources (images, fonts, etc) and even multiple threads to run the gui. On my Mac, I once counted 43 different threads active when launching Sigil with a large book. Part of the reason for the large number of threads is that my cpu has 4 actual cores and therefore 8 logical hypercores.

Windows 10, version 1803, I'm not sure what update number it is but I have only had two failed updates.

Brent

[brent63]

Originally Posted by KevinH

One interesting test just before you go to bed at night would be to:

1. close the Preview window and exit from Sigil (to prevent the Preview Window from reopening)
2. restart Windows 10
3. start a fresh copy of Sigil and open the latest version of our User's Guide (so we are all starting with exactly the same epub). Verify that Preview is NOT open.
4. Then check and write down the memory usage of Sigil
5. Then close the lid to sleep for the night

Once you get up the next day.

6. Reopen the lid and **without** clicking on Sigil (which would start new gui threads) check the memory and write it down.

Let us know what if anything changed during sleep.

Then repeat the entire process the next night but with Preview ***Open***.

That way we can see how much memory consumptions changes overnight with and without Preview being open.

This may help isolate any increased memory usage to the Qt Webkit or not.

Thanks!

KevinH

Okay, will try it tonight.

Brent

[DiapDealer]

Originally Posted by brent63

I have 4 gig of ram, can max it out to eight, but this is the only program showing any slowdown.

Understood. But to be fair, you probably don't have many other programs that: are embedding Python; are using the (nearly) very latest Qt; include a built-in web browser.

[DiapDealer]

Originally Posted by brent63

Windows 10, version 1803, I'm not sure what update number it is but I have only had two failed updates.

I'm beginning to suspect that W10 1803 and W8.1 are going to be those hardest hit by Sigil 0.9.11+ performance issues. I'm not sure anything other than 1809 has received the partial "fixes" for the delays that were introduced by Control Flow Guard.

Another thing to try is to disable CFG for the Sigil binary. I can't vouch for Windows 8.1 but I know that Windows 10 (1803 and 1809) can override the system-wide security settings on a program by program basis. In fact, Windows has already overridden individual security settings for several binaries.

From Windows Security, click on App & browser Control; scroll down to Exploit protection and click the "Exploit protection settings" link. There's System overrides (that I do not recommend changing unless you're sure of what you're doing), and there's Program overrides. You can add Sigil.exe (by name or by explicit path) and then edit the individual overrides you want to affect Sigil. Control Flow Guard is on by default and you can override that and turn it off for Sigil only.

You can easily reverse that override, or remove Sigil entirely from the program list once done testing.

[KevinH]

Originally Posted by brent63

I have opened widget at the bottom of the preview page, and gone thru the seven items there but don't see any issues?

Can you be more specific as to where to look?

One of the tabs shows a folded html tree that shows the tree as built insideof Qt. By unfolding tags by pressing on the triangle you should be able to navigate to where the italic i tag is and verify it exists and that no css is impacting it, etc.

[brent63]

Originally Posted by KevinH

One of the tabs shows a folded html tree that shows the tree as built insideof Qt. By unfolding tags by pressing on the triangle you should be able to navigate to where the italic i tag is and verify it exists and that no css is impacting it, etc.

The italic i tags are there.

I opened a file where I'm not embedding the same font and the italic tags work fine.

I tried to embed the Georgia font directly from the windows font folder but ran in to permission issues.

When I copied and pasted it to current work folder the font created 4 versions of itself,

Normal
Bold
Italic
ItalicBold

I embedded all of them but the Italic is not showing

Normal shows because it's linked to <p>

Bold shows because it's linked to <h1>

ItalicBold shows because it's linked to <h2>

Brent

[KevinH]

So it is probably a Qt issue with that particular font. Some fonts include both unix and windows font tables but only the unix font tables are correct. Qt is one of the very few programs left that uses the older Windows specific OS2 font tables that are often broken or incorrect in modern fonts because they are so rarely used.

Even on Mac, if I repair a font using a font utility the older OS2 font tables are generally ignored. That leaves some fonts as broken under Qt even though Qt follows the OS2 font table spec on Windows.

[DiapDealer]

I think its probably worth a test to see if Sigil exhibits the same extreme performance issues when editing an epub that doesn't embed any fonts. Because I'll be honest: I can see this version of Sigil getting seriously bogged down (on Windows especially) if every single character of the book is being displayed with an embedded font. Using an embedded font for normal body text is simply not done that much these days.

[KevinH]

For the record: See this Windows specific bug in Qt that caused some fonts to be detected as italic when not, and other bugs that were italic to be detected as normal:

https://bugreports.qt.io/browse/QTBUG-73660

The fix for this bug will be in Qt 5.12.2 and later which we will be using for our next release.

KevinH

Originally Posted by KevinH

So it is probably a Qt issue with that particular font. Some fonts include both unix and windows font tables but only the unix font tables are correct. Qt is one of the very few programs left that uses the older Windows specific OS2 font tables that are often broken or incorrect in modern fonts because they are so rarely used.

Even on Mac, if I repair a font using a font utility the older OS2 font tables are generally ignored. That leaves some fonts as broken under Qt even though Qt follows the OS2 font table spec on Windows.

[theducks]

FWIW
I just rolled back to 9.9 from 9.13 after experiencing slower, then slower, then slower after running a lot of S&R's and spell check fixes.
We are talking under an Hour Sigil sessions. Restarting Sigil goes back to intital speeds (still slower than 9.9) With 9.13 you SEE find next switch files, then scroll

I was not seeing massive memory growth during this time (I have a resource display on my Logitec G3. Crude...) Something odd has crept into the 9.13 code
W10 x64 1803 8G RAM

[DiapDealer]

Originally Posted by theducks

Something odd has crept into the 9.13 code

W10 x64 1803 8G RAM

No. It really hasn't.

Something odd is going on between newer versions of Qt/QtWebkit and Windows 10. Case in point: I'm rocking the same code as you on W10 x64 1809 with 8G RAM and not a problem in the world. So the code itself is just fine. Something else is not.

[BetterRed]

Originally Posted by theducks

FWIW
I just rolled back to 9.9 from 9.13 after experiencing slower, then slower, then slower after running a lot of S&R's and spell check fixes.
We are talking under an Hour Sigil sessions. Restarting Sigil goes back to intital speeds (still slower than 9.9) With 9.13 you SEE find next switch files, then scroll

I was not seeing massive memory growth during this time (I have a resource display on my Logitec G3. Crude...) Something odd has crept into the 9.13 code
W10 x64 1803 8G RAM

Did you try disabling CFG for Sigil, as per post #110.

BR

[theducks]

Originally Posted by BetterRed

Did you try disabling CFG for Sigil, as per post #110.

BR

No I hadn't.

That was fun, figuring out that I had to click what looked like a section title

[BetterRed]

Originally Posted by theducks

That was fun, figuring out that I had to click what looked like a section title

Apart from getting yourself sectioned did disabling it overcome the performance issues?

BR