Sigil-0.9.13 Released - Page 2

BetterRed · 03-22-2019, 04:58 PM

Quote:

Originally Posted by DiapDealer

Can you exclude Sigil's program directory and the %localappdata%\sigil-ebook folder from all anti-virus, anti-malware scanners?

@icallaci - and is there any chance that an anti-whatever scanner, or an always-on backup service (OneDrive, Dropbox etc) is accessing the directory in which Sigil puts its working files - this can be set in Preferences->General->Set Folder where temporary files should be created. By default they are in %APPDATA%\Local\Temp\Sigil-Xxxxxx - where Xxxxxx is random.

BR

icallaci · 03-22-2019, 07:09 PM

Excluding Sigil and its program and working directories from the antivirus made no difference. There are no cloud services at all installed. I have two other computers in the house, so I tried those as well. One is a Windows 8.1 laptop and the other is a Windows 7 desktop (both with a different antivirus installed). Both computers show the same behavior, with increasing memory usage and slowdowns. I am beginning to suspect that either I use Sigil for longer sessions than most users, or else something about the regexes I run are using up system resources. At this point, if no one else is having trouble, I am willing to concede that it is something I am doing. Also, it is not really a problem for me to close Sigil and reopen it, because that sets everything back to normal and I can continue working without problems for a while. Thanks for looking into this. If I can help in some other way, I'll be glad to try; otherwise, go have yourself a beer and stop worrying about this.

KevinH · 03-22-2019, 07:38 PM

Quote:

Originally Posted by icallaci

. I am beginning to suspect that either I use Sigil for longer sessions than most users, or else something about the regexes I run are using up system resources.

It could be a regex related search and replace that has a memory leak we simply have not touched on. For a leak to show up in Valgrind or macOS "leaks tool" you have to actually exercise the problem code to detect a leak.

So it would really help us if you could please post a sample epub and a list of regex expressions you run typically and the order. That way we can try the exact samething you are doing in a leak detector to see if memory really is leaking. Please try to include an exact set of steps that seems to use the most memory.

If you prefer, you can pm DiapDealer and KevinH here with the link to the epub and list of regexes.

I really would like to get this tracked down and fixed.

Thanks,

KevinH

KevinH · 03-22-2019, 07:48 PM

Also can you please confirm this same issue does NOT happen with Sigil 0.9.10 or Sigil 0.9.9?

Or could it have existed all along but only now you can recreate it more easily?

That info would help narrow things down, as nothing in the Search and Replace code has changed since those releases, just Python (which is not used here) and Qt have had significant code changes.

Thanks!

KevinH

icallaci · 03-22-2019, 09:34 PM

I'm 99% sure this didn't happen with 0.9.10 or 0.9.9 because it's only been an obvious problem with the latest releases. I haven't changed my workflow in ages, but it's possible that there was a problem that just didn't surface as quickly with the earlier releases. If and when I get a block of time, I'll uninstall 0.9.13 and go back to 0.9.10 to see what happens. That may take me a few days to get to, though. In the meantime, I'll post my regexes so you can take a look at them.

Quote:

Originally Posted by KevinH

Also can you please confirm this same issue does NOT happen with Sigil 0.9.10 or Sigil 0.9.9?

Or could it have existed all along but only now you can recreate it more easily?

That info would help narrow things down, as nothing in the Search and Replace code has changed since those releases, just Python (which is not used here) and Qt have had significant code changes.

Thanks!

KevinH

icallaci · 03-22-2019, 09:38 PM

I've attached an .rtf with all the regexes I run, in order (and notes to myself about what each regex is supposed to do). It's quite tedious, lol, but I'd rather fix things up front than be annoyed by problems in the middle of reading. The memory leak shows up most quickly in non-fiction books with lots of formatting, like footnotes, index entries, bibliographies with a href links, etc. The more complicated the better.

Quote:

Originally Posted by KevinH

It could be a regex related search and replace that has a memory leak we simply have not touched on. For a leak to show up in Valgrind or macOS "leaks tool" you have to actually exercise the problem code to detect a leak.

So it would really help us if you could please post a sample epub and a list of regex expressions you run typically and the order. That way we can try the exact samething you are doing in a leak detector to see if memory really is leaking. Please try to include an exact set of steps that seems to use the most memory.

If you prefer, you can pm DiapDealer and KevinH here with the link to the epub and list of regexes.

I really would like to get this tracked down and fixed.

Thanks,

KevinH

KevinH · 03-22-2019, 10:04 PM

Great! Thank you! I will run full memory leak detection using these.

Also, can anyone point me to an epub like the one described by icallaci above in our own MR library or somewhere in the public domain to test with?

Thanks

Quote:

The memory leak shows up most quickly in non-fiction books with lots of formatting, like footnotes, index entries, bibliographies with a href links, etc. The more complicated the better.

icallaci · 03-22-2019, 10:19 PM

I'm sorry I don't have an epub for you to run your tests against. Once I fix them, I delete the originals.

Quote:

Originally Posted by KevinH

Great! Thank you! I will run full memory leak detection using these.

Also, can anyone point me to an epub like the one described by icallaci above in our own MR library or somewhere in the public domain to test with?

Thanks

Tex2002ans · 03-23-2019, 01:51 AM

Quote:

Originally Posted by icallaci

I'm still experiencing major slowdowns

Is anyone else getting a really slow-to-finally-popup Spellcheck List?

Compared to Sigil 0.9.10, running Tools > Spellcheck > Spellcheck is noticeably slower. (Same EPUB ~<1 second in 0.9.10 to 6+ seconds in 0.9.13).

Windows 10, 64-bit.
Sigil 0.9.13 64-bit.

I rolled back my Sigil install + sigil.ini to 0.9.10 when the .12 trouble happened, so this was a straight 0.9.10->0.9.13 upgrade.

Note: I'll be sure to test on more books... this was just using the latest EPUB I've been working on for a few months, which is why I found it very noticeable.

theducks · 03-23-2019, 05:34 AM

Funny you should ask.
I noticed almost every GUI state change seems a bit sluggish (Open a new tab, a dialog open...). The rest of the task runs at speed.
W10 x64 on a Intel Core Duo w/8G RAM (certified to NOT to be a speed demon

)

DiapDealer · 03-23-2019, 07:01 AM

Even if you're using what you believe to be "clean" preferences, please back them up and make sure you can reproduce your symptoms with freshly generated ones.

Also make sure you're using the very latest driver available for your video card.

BetterRed · 03-23-2019, 07:53 AM

FWIW - I installed Windows 64 bit 9.13 over 9.9 early this morning, been using it on and off all day in one to two hour sessions. If anything, my impression is that it's snappier, including - even especially - the spell checker, and reports

Thanks for the extra Open With slots.

Added, up-to-date Windows Pro 64 bit version 1809, Dell XPS 1890, i7-7700 CPU, 16Gb RAM, NVIDIA GeForce GTX 1060 6GB, with driver version 23.21.13.9125

BR

icallaci · 03-23-2019, 09:35 AM

KevinH, I was going to PM you an epub that might help, but it appears I can't attach a file to a PM, and I don't think it matters what book you use anyway. Also, after thinking about this during the night (when I SHOULD have been asleep, lol), the problem may not be related to regexes at all. I can actually cause this problem to happen just by opening and closing the Preview window, or doing a spellcheck, or opening and closing individual .html files in the epub. It takes quite a bit longer for the slowdown to occur, but if I watch the memory in Task Manager, it climbs steadily. Honestly, if you can't track this down, it's not a problem for me to simply close Sigil and reopen it, especially since most people are not experiencing this problem. I must be special

DiapDealer · 03-23-2019, 10:06 AM

My big concern is that it seems to be happening for you on multiple machines (with different OSes). 0.9.13 is actually snappier for me on both Win7 and Win10 than previous versions. Url-filled epubs with many, images and many xhtml files (all open simultaneously) rarely hits the 270mb mark of memory usage. The 32-bit version often runs at under 100Mb for me with "average" epubs.

KevinH · 03-23-2019, 11:07 AM

Also, please understand the memory footprint will increase and decrease at times as it builds caches and things. With large books I can get the memory footprint to approach 500 meg but never the 1 gig and over that you are seeing.

That said, the issue is not the memory footprint, it is that this memory growth actually impacts your performance of the app in that is causes massive slowdowns and crashes.

Does this Sigil-0.9.13 still cause huge slowdowns and crashes that you reported earlier? Or is it just that the memory footprint seems to grow and not shrink for you.

Thanks,

KevinH

03-23-2019, 07:53 AM	#27
BetterRed null operator (he/him) Posts: 20,568 Karma: 26954694 Join Date: Mar 2012 Location: Sydney Australia Device: none	FWIW - I installed Windows 64 bit 9.13 over 9.9 early this morning, been using it on and off all day in one to two hour sessions. If anything, my impression is that it's snappier, including - even especially - the spell checker, and reports Thanks for the extra Open With slots. Added, up-to-date Windows Pro 64 bit version 1809, Dell XPS 1890, i7-7700 CPU, 16Gb RAM, NVIDIA GeForce GTX 1060 6GB, with driver version 23.21.13.9125 BR Last edited by BetterRed; 03-23-2019 at 08:09 AM.

03-23-2019, 10:06 AM	#29
DiapDealer Grand Sorcerer Posts: 27,549 Karma: 193191846 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	My big concern is that it seems to be happening for you on multiple machines (with different OSes). 0.9.13 is actually snappier for me on both Win7 and Win10 than previous versions. Url-filled epubs with many, images and many xhtml files (all open simultaneously) rarely hits the 270mb mark of memory usage. The 32-bit version often runs at under 100Mb for me with "average" epubs. Last edited by DiapDealer; 03-23-2019 at 10:09 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Sigil-0.9.5 Released	KevinH	Sigil	68	04-09-2016 06:30 AM
Sigil-0.9.1 Released	KevinH	Sigil	36	12-04-2015 03:00 PM
Sigil-0.8.900 released for testing - Wait for Sigil-0.8.901	KevinH	Sigil	106	10-04-2015 10:41 AM
Sigil 0.8.3 Released	user_none	Sigil	10	02-02-2015 04:32 PM

03-22-2019, 07:09 PM	#17
icallaci Guru Posts: 768 Karma: 6528026 Join Date: Sep 2012 Device: Kobo Elipsa	Excluding Sigil and its program and working directories from the antivirus made no difference. There are no cloud services at all installed. I have two other computers in the house, so I tried those as well. One is a Windows 8.1 laptop and the other is a Windows 7 desktop (both with a different antivirus installed). Both computers show the same behavior, with increasing memory usage and slowdowns. I am beginning to suspect that either I use Sigil for longer sessions than most users, or else something about the regexes I run are using up system resources. At this point, if no one else is having trouble, I am willing to concede that it is something I am doing. Also, it is not really a problem for me to close Sigil and reopen it, because that sets everything back to normal and I can continue working without problems for a while. Thanks for looking into this. If I can help in some other way, I'll be glad to try; otherwise, go have yourself a beer and stop worrying about this.

03-22-2019, 07:48 PM	#19
KevinH Sigil Developer Posts: 7,644 Karma: 5433388 Join Date: Nov 2009 Device: many	Also can you please confirm this same issue does NOT happen with Sigil 0.9.10 or Sigil 0.9.9? Or could it have existed all along but only now you can recreate it more easily? That info would help narrow things down, as nothing in the Search and Replace code has changed since those releases, just Python (which is not used here) and Qt have had significant code changes. Thanks! KevinH

03-23-2019, 05:34 AM	#25
theducks Well trained by Cats Posts: 29,800 Karma: 54830978 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	Funny you should ask. I noticed almost every GUI state change seems a bit sluggish (Open a new tab, a dialog open...). The rest of the task runs at speed. W10 x64 on a Intel Core Duo w/8G RAM (certified to NOT to be a speed demon )

03-23-2019, 07:01 AM	#26
DiapDealer Grand Sorcerer Posts: 27,549 Karma: 193191846 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	Even if you're using what you believe to be "clean" preferences, please back them up and make sure you can reproduce your symptoms with freshly generated ones. Also make sure you're using the very latest driver available for your video card.

03-23-2019, 09:35 AM	#28
icallaci Guru Posts: 768 Karma: 6528026 Join Date: Sep 2012 Device: Kobo Elipsa	KevinH, I was going to PM you an epub that might help, but it appears I can't attach a file to a PM, and I don't think it matters what book you use anyway. Also, after thinking about this during the night (when I SHOULD have been asleep, lol), the problem may not be related to regexes at all. I can actually cause this problem to happen just by opening and closing the Preview window, or doing a spellcheck, or opening and closing individual .html files in the epub. It takes quite a bit longer for the slowdown to occur, but if I watch the memory in Task Manager, it climbs steadily. Honestly, if you can't track this down, it's not a problem for me to simply close Sigil and reopen it, especially since most people are not experiencing this problem. I must be special

03-23-2019, 11:07 AM	#30
KevinH Sigil Developer Posts: 7,644 Karma: 5433388 Join Date: Nov 2009 Device: many	Also, please understand the memory footprint will increase and decrease at times as it builds caches and things. With large books I can get the memory footprint to approach 500 meg but never the 1 gig and over that you are seeing. That said, the issue is not the memory footprint, it is that this memory growth actually impacts your performance of the app in that is causes massive slowdowns and crashes. Does this Sigil-0.9.13 still cause huge slowdowns and crashes that you reported earlier? Or is it just that the memory footprint seems to grow and not shrink for you. Thanks, KevinH

Advert

Advert