Suggestions for Sigil Improvements or New Features

[KevinH]

And even with the QSG_RHI_BACKEND=opengl set in env vars, I still occaissionally get the damn GetCaretLocation to return a null string even when the timeout is longer.

So something on Windows, related to graphics backend is causing the load race, all because we are properly clearing the cache now.

Very strange.

[KevinH]

I have tried a different approach and pushed it to master with skip ci and then will manually run the Windows workflow.

I really hate Windows btw. I had to download my Sigil installer 3 times until I figured out how to tell Windows 11, I decide which downloads to keep not Microsoft. And why on earth is Microsoft interfering with downloads from Releases on a github owned site given they own github!

Arrrrgggghhhhhhh!

[DNSB]

On my main computer, I get the same pop back to the top of preview when I open the stylesheet. I tried the QSG_RHI_BACKEND env var but it seemed to make no difference.

I'm going to download your latest code and try that. Gives me something to do during the commercial breaks in the hockey game.

[DNSB]

Still goes back to the top of file in preview when I open the stylesheet.

[DiapDealer]

For what it's worth... I've only been able to verify that QSG_RSI_BACKEND=opengl makes a consistent difference on one Windows 10 virtual machine.

[KevinH]

In fact, the more I test different back ends the % loaded bar underneath Preview gets stuck at 70% or 80% and sometimes even after a manual reload it is stuck at 10% or 30%.

So something very strange is happening.

My latest change in master seems to help but it is not a full solution.

I will keep digging.

[KevinH]

Originally Posted by DNSB

Still goes back to the top of file in preview when I open the stylesheet.

Your log file seems to show that somehow when doing a reload after clearing the cache, Preview on Windows is not properly loading consistently.

I am seeing something similar in my tests.

[KevinH]

Okay, I have cleaned up a few things and then added the complete ViewPreview load debug output. Hopefully this will show us what is happening during all loads and reloads of Preview on Windows.

On MacOS and Linux, the current master seems to work just fine.

So whatever is happening it is specific to Windows and relates to full cache clearing, load vs reloads of Preview, backend used, and GetCaretLocation call in javascript failing (called to early).

[KevinH]

Okay, I think I have tracked down the issue. Starting a few releases back in Qt clearing the web cache became an asynchronous operation. So we allowed qApp->processEvents to happen while we looped until the cache signalled it was empty. This allowed other gui events to be processed including starting to load a page in Preview before the cache is finally fully cleared.

This resulted in mayhem.

For some reason, clearing the cache just takes so much longer on Windows than it does on MacOS and Linux.

So almost 100% of the time, on MacOS and Linux you will see a debug message that cache clear started, and immediately following it cache clear ending with no intervening events.

But on Windows, it takes so long that cache clearing allows other events to happen and things get out of sync on Windows resulting in unfinished loads and reloads happening which prevents that javascript call to GetCaretLocation from happening properly.

Not really sure how to handle this as the cache must be fully cleared before we start to load Preview as some CSS files, and SVG files need have been updated (ie. you editing a css file).

So somehow we must wait and block event processing as moving forward just does *not* work. This is always going to be a problem when mixing synchronous code with asynchronous code. Wedkit was fully synchronous. QtWebEngine if fully asynchronous.


I have tried setting the WebEngineProfile to NoCache but it still seems to cache things.

This is going to take some time.

[DNSB]

If it's any help, here is my log file from running the latest code. Oddly, I noticed that when I opened the stylesheet, Preview went to the top of the file. While I was typing in the stylesheet adding the body { color: red; }, Preview refreshed a few more times with some of them going to the spot where I had clicked but most still landing at the top of the file.

Edit: the test was run with my standard setup. I then added QTWEBENGINE_CHROMIUM_FLAGS="--disable-gpu" and QSG_RHI_BACKEND=opengl to the environmental variables and the only change noticed was that Preview never went back to the correct location.

[KevinH]

Originally Posted by DNSB

If it's any help, here is my log file from running the latest code. Oddly, I noticed that when I opened the stylesheet, Preview went to the top of the file. While I was typing in the stylesheet adding the body { color: red; }, Preview refreshed a few more times with some of them going to the spot where I had clicked but most still landing at the top of the file.

Edit: the test was run with my standard setup. I then added QTWEBENGINE_CHROMIUM_FLAGS="--disable-gpu" and QSG_RHI_BACKEND=opengl to the environmental variables and the only change noticed was that Preview never went back to the correct location.

Thanks,

Here is the problem in a nutshell from you log file:

Debug: clearing Preview's httpcache
Debug: Loading a page started
Debug: Loading progress  0
Debug: linkHovered received  ""
Debug: Loading progress  100
Debug: UpdateFinishedState with okay  true
Debug: WebPageJavascriptOnLoad start
Debug: WebPageJavascriptOnLoad end
Debug: WebPageJavascriptOnLoad with m_CustomSetDocumentInProgress:  true
Debug: ViewPreview Cache Cleared

We start to clear the httpcache but before it fully clears, we have already started reloading Preview, and not just started it and in this instance we have loaded the entire thing BEFORE our cache clear is ever finished.

So we are adding files to the ceche while we are simultaneously clearing it.

I think the only way to handle this is to split our custom load into two pieces with the first piece just clearing the cache which invokes the seond piece only when it is complete in a callback.

UUUUUggggghhhhhhhh!

[DNSB]

So the few times in my first test that the Preview went back to the correct locations was when the log file showed:

Debug: clearing Preview's httpcache
Debug: Loading a page started
Debug: Loading progress  0
Debug: ViewPreview Cache Cleared

Even there it shows the loading started before the cache returned as cleared.

[KevinH]

But little to nothing was loaded.

I am going to have to really redesign the cache clearing approach. Furthermore based on testing on a Mac I will need to redesign the WebProfileMgr class to make each MainWindow use its own profile.

This change by Qt to use asynchronous cache clearing is really hard to workaround without a structural redesign.

And I tried following Qt guide to disable all caching by setting NoCache in the profiles but it just doesn't work. Internal caches are still being kept.

An interesting redesign process. Mixing asynchronous code with synchronous code is really for the birds!

[KevinH]

Okay, I took a shot at making ViewPreview more asynchronous and changed how the caches are cleared to minimize load and cache clear overlap. Had to redesign WebProfileMgr to use unique data names and to always give new profiles (no reuse) so that clearing a profile cache did not fry second instance of Sigil.

I have pushed all of this to master. So if you build your own, please give a try and let me know if it works and what the log file shows.

I have also made a ci build for just Windows. I have uploaded it to my personal repo (see earlier links) and the file is names Sigil-2.7.5-DEBUG5-Windows-x64-Setup.exe
in case you have time to help out but do not build your own.

Thanks everyone for all your help! It is greatly appreciated.

[DNSB]

Just checked the preview window and the cursor remained where I placed it as the preview window refreshed multiple times. Looks good!

Edit: I loaded @RbnJrg's filter test epub and it seems to show no issues. Changed the colour in the filter file from red to rebeccapurple and the refresh switched the colour. Ditto from rebeccapurple to olivedrab to peachpuff.