Sigil running multiple instances

[DiapDealer]

I see a lot of scattering of webengine cache files on both Linux and Windows. I've always meant to try Kovid's approach at unifying on Windows (requires patching qtbase), but I've never gotten around to it.

https://github.com/kovidgoyal/bypy/b...qt_base.py#L66

https://github.com/kovidgoyal/calibr...stants.py#L423

I don't see anything similar for Linux or macOS in calibre, so I'm assuming there might be another way to consolidate that doesn't require patching?

Either that, or the default qt cache locations on Linux or macOS don't cause any issues.

On Arch Linux, I have a QtWebEngine folder in ~/.cache/sigil-ebook/sigil

Inside that QtWebEngine folder is a folder named 'Preview' that contains cache data that is recreated if I delete the folders and relaunch Sigil. There was also a 'Default' folder adjacent to 'Preview' that has not been recreated thus far. It could be a remnant from something past.

The standard Sigil Preference directory also contains a QtWebEngine folder with a Preview folder in it (that gets recreated when deleted and relaunching Sigil), but that one does not appear to get any new cache data written to it beyond that (at least not with the content I've opened thus far).

Also note that I have no issues running multiple instances of Sigil on Linux when using a build of the latest git commit. It seems to be a Windows only thing.

I'll try to check Windows in a bit.

[DiapDealer]

On Windows:
Windows: C:/Users/<USER>AppData/Local/sigil-ebook/sigil/ IS the standard Sigil prefs folder. C:/Users/<USER>AppData/Local/sigil-ebook/sigil/cache/QtWebEngine exists (with cache data under 'Preview'), and it gets recreated when deleted and Sigil is relaunched.

There was also a QtWebEngine folder without the 'cache' prefix in the prefs folder (C:/Users/<USER>AppData/Local/sigil-ebook/sigil/QtWebEngine) but it does not get recreated after deleting. I fear older versions of Qt wrote to different locations).

There is also a 'QtWebEngine' folder under C:/Users/<USER>AppData/Roaming/sigil-ebook/sigil that contains an empty 'Preview' folder that gets recreated when deleted, but doesn't appear to have any actual cache data being written to it.

So on both Linux and Windows a 'Preview' cache folder is being created that doesn't appear to have any cache data being written to it. At least not always.

[KevinH]

Okay I have tried something that may not help but it is worth a shot.

I have modified ViewPreview to use a deadline timer to prevent an infinite loop and then modified our WebProfileMgr to use a memory based cache that maybe can be shared.

I have pushed these changes to github Master.

So DNSB and DiapDealer, please give that a try on Windows and let me know if it helps when trying to use multiple instances of Sigil at the same time.

You will need to turn on the DEBUG log file in Windows via our environment variable:

SIGIL_DEBUG_LOGFILE set be a full path including file name to a "debug_sigil.txt" file on your Desktop (or someplace else you know where to look and have write access to) and look for this message in the output:

qDebug() << "View Preview Cache Clear failed - deadline expired";

That message indicates that the Cache clear failed and so we need to try a different approach.

I put that deadline timer in just to prevent a infinite loop.

[KevinH]

Interestingly, there are so many auto created QtWebEngine storage locations, that in Qt 6.9.0 they have created a whole new QtWebEngineProfileBuilder class to help create them only once and where you want them.

See https://doc.qt.io/qt-6/qwebengineprofilebuilder.html

So once we move to Qt 6.9.0 we can try that.

[DiapDealer]

I'm getting the "deadline expired" message. I opened two instances of Sigil and opened a different epub in each of those instances.

Interestingly... I'm getting a new warning message about Sigil not being able to find any webengine spellcheck dictionaries (not that we use them in Sigil).

Warning: The following paths were searched for Qt WebEngine dictionaries:
  C:/Program Files/Sigil\qtwebengine_dictionaries
  C:/Program Files/Sigil\qtwebengine_dictionaries
but could not find it.
You may override the default search path by using QTWEBENGINE_DICTIONARIES_PATH environment variable.
Spellchecking can not be enabled.

Seems harmless enough, but has something changed in our profile for WebEngine to cause that? Or is it because of the switch to a memory cache instead of a disk one?

EDIT: I can see where you explicitly set setSpellCheckEnabled to false in the default profile in your commit, so something else must be going on.

[DNSB]

Much the same as @DiapDealer reported. I've attached my sigil_debug.txt file to this message though it's almost identical.

[DNSB]

Just for the heck of it since the page-edit install directory had a qtwebengine_dictionaries directory in it, I tried copying it to the Sigil directory. The errors related to the qtwebengine_dictionaries disappear but I'm not sure what other effects this may have.

[KevinH]

Yes, both of you showed the timer expired message so making it a MemoryCache did not help.

Okay, I think I am going to go back to disk caches but upon exiting Sigil, I will have it erase its own cache so lots of unique storage names are not left to fill things up. That should work.

I will push something to master when I have something for you to test.

And Yes, setting the profile to setSpellCheckEnabled to false was an attempt to stop those silly error messages (they appear on MacOS as well) but it seems to no avail. I think they are a harmless warning.

[KevinH]

Okay, I have implemented unique disk cache storage names and cleanup on export and pushed it to master.

If either of you get a chance (after removing the old debug log txt file), please try with current master and multiple instances of Sigil and see if you still get the expired timer debug message.

If this does not work, then the locking is happening someplace else that impacts the cache clearing, not in the Preview storage name.

If that happens, we will try going with no storage name at all for Preview so it becomes an off-the-record style profile.

At worst case we can skip the cache flushing but that would be a last resort.

I have my fingers crossed.

[DiapDealer]

I'm still getting the deadline expired message. I can see the unique folder names ('Preview + uuid') getting created, but they're being created in C:/Users/<USER>/AppData/Roaming/sigil-ebook/sigil and they are empty of content. Folders only. After all instances of Sigil are closed, the empty 'Preview + uuid' folders are left behind.

At the same time, a new folder called Preview-Cache is being created in Sigil's normal prefs folder ( C:/Users/<USER>/AppData/Local/sigil-ebook/sigil) which does contain content. One folder only -- no matter how many Sigil instances.

It's almost as if the reading/writing and cleanup are trying to access two different locations and getting confused as to what's where.

[KevinH]

The fact that the empty folders are left behind is in fact indicative of a real problem. They should have been erased when Sigil exited. It is in main.cpp right at the end.

So something is definitely funny. Everything should now be moved to be inside the Sigil Prefs folder.

On Windows is Sigil Prefs in roaming?

They should be appearing inside the QtWebEngine folder right beside where the local-storage and PreviewCache folders are created.

Perhaps we do need to somehow patch Qt on Windows to get the cache path set to be someplace normal.

Either way, something is blocking the clearing of the cache, but they should now be in separate unique folders and no file lock should interfere.

I think you may be right and something in chromium/qtwebengine is creating the caches in one place while trying to clear the cache someplace else.

So very strange.

I am going to try just using unnamed storage and making off-the-record profile for Preview to see if that changes anything. I will push it to master once I get something working.

[DiapDealer]

No. Sigil's prefs are in Local on Windows.

[DiapDealer]

The QtWebEngine folder is not being created in either location (Roaming or Local) right now. I see no check for it to be created if missing in the current code.

[KevinH]

Okay just pushed to master the "off the record" approach for Preview so that it does not use a storage name at all.

I really hope this works.

[KevinH]

BTW: what determines if the Qt AppDataLocation maps to "local" vs "roaming" on Windows? Is it user configurable?