Slow switching of Views - Long thread

[JimmXinu]

I was thinking that it might be more useful to BR or chaley actually, since they know more about library structure than I do. I was unclear.

That said, I did download it and try it in my Win10 Cal5.38 64bit on an SSD. Switching between VL or directly between VM views with this (nearly) empty test library is basically instant for me.

jnikkir's CPU (reported earlier) is a 2017 model according to google, that shouldn't be old enough to make that much difference. Not unless win11 on that hardware is a slug. I remember past windows 'upgrades' that made older hardware basically unusable.

If it were me, I'd try reinstalling Calibre (and making sure you have 64bit) and rebooting the computer if you haven't already. Not because I have a reason, just because it's a reflex.

[ownedbycats]

I would also consider checking hard drive health and running a memtest.

[BetterRed]

Originally Posted by JimmXinu

I was thinking that it might be more useful to BR or chaley actually, since they know more about library structure than I do. I was unclear.

@jnikkir - feel free to PM the library to me, but in light of the following I doubt i can discover much. What I can do is run a test in safe mode, I am still scratching my head over the error you got - that just shouldn't happen.

Originally Posted by JimmXinu

That said, I did download it and try it in my Win10 Cal5.38 64bit on an SSD. Switching between VL or directly between VM views with this (nearly) empty test library is basically instant for me.

jnikkir's CPU (reported earlier) is a 2017 model according to google, that shouldn't be old enough to make that much difference. Not unless win11 on that hardware is a slug. I remember past windows 'upgrades' that made older hardware basically unusable.

Apparently my i7-7700 3.60GHz 16G RAM with a 256GB SSD isn't good enough to run Win11.

Originally Posted by JimmXinu

If it were me, I'd try reinstalling Calibre (and making sure you have 64bit) and rebooting the computer if you haven't already. Not because I have a reason, just because it's a reflex.

I'd also try renaming the calibre configuration folder from ..\calibre to ..\calibre-save (calibre not running) and then letting calibre rebuild a fresh configuration - I can do a test of that if @jnikkir sends me the library.

And what owb suggests.

BR

[jnikkir]

Okay, so many suggestions -- thank you!! They took me some time to get through, but here's what I've done:

Originally Posted by ownedbycats

I would also consider checking hard drive health and running a memtest.

Checked my hard drive health with that app, got a Status: Green. I am a little shy about running a memtest -- I'll have to look into a how-to for that.

Originally Posted by BetterRed

I'd also try renaming the calibre configuration folder from ..\calibre to ..\calibre-save (calibre not running) and then letting calibre rebuild a fresh configuration

Tried this -- same lag.

Originally Posted by JimmXinu

That said, I did download it and try it in my Win10 Cal5.38 64bit on an SSD. Switching between VL or directly between VM views with this (nearly) empty test library is basically instant for me.

::laughcries::

Originally Posted by JimmXinu

jnikkir's CPU (reported earlier) is a 2017 model according to google, that shouldn't be old enough to make that much difference. Not unless win11 on that hardware is a slug. I remember past windows 'upgrades' that made older hardware basically unusable.

Definitely not a slug. Sometimes it's a little slow if I'm dealing with really large files in Photoshop or Illustrator, or when I have a LOT of programs open at once. But Calibre is usually very fast, and I've run it along w/ no other programs open and it still lags...

Originally Posted by JimmXinu

If it were me, I'd try reinstalling Calibre (and making sure you have 64bit) and rebooting the computer if you haven't already. Not because I have a reason, just because it's a reflex.

Done... no dice. I'm pretty sure I tried that before, but honestly I've tried so much it's all a blur.

Gonna PM my library to BR....

[BetterRed]

jnikkir's library is OK in Safe Mode here - maybe a Win 10 v 11 thing.

I put the library on my 1TB internal SATA HDD - the VM switch times on the library (13 books and a single sort column) were laggy - much slower than VM switch times for bigger libraries with multiple sort columns on the same device.

But here's the rub, VM switching on jnikkir's library was also rattly - I could hear disk r/w heads moving - that's not happening on my bigger, and arguably more complex libraries.

On a whim, I put a copy of the library onto my 256GB SSD - result, VM switching near instantaneous. I admit that I've never been an SSD fan, not because there's anything wrong with them. But I'm not a speed freak and the claims for SSDs have IMO been over-hyped --- but this is the most dramatic effect I've seen from an SSD.

So, I tried moving the metadata.db file from the library folder on the rattly drive to the SSD and dropping a symlink to it in the library folder on the rattly drive. Guess what, less laggy and no rattling. Which seems to indicate that the disk I/O I can hear on the rattly drive is related to the metadata.db file - which surprises me.

I can't figure out why this tiddly little library should be rattling the disk drives cage but there's no doubt it is.

BR

[ownedbycats]

I'd be curious to see what FileActivityWatch reports.

[chaley]

Originally Posted by jnikkir

Gonna PM my library to BR....

If you send both the library and the calibre configuration folder (zipped up) to me then I will take a look. However, all I have is an SSD so given BetterRed's experience I might not see the problem.

Have you tried running Library / Library Maintenance / Check library? It is possible that the database is very fragmented.

[BetterRed]

Originally Posted by chaley

If you send both the library and the calibre configuration folder (zipped up) to me then I will take a look. However, all I have is an SSD so given BetterRed's experience I might not see the problem.

Have you tried running Library / Library Maintenance / Check library? It is possible that the database is very fragmented.

I did a Library Check - made no difference.

Still quite laggy on a rather old Flash thumb drive - no sound effects though

BR

[BetterRed]

And I tried it in a vanilla portable + View Manager plugin installed on the HDD - still red rattler and laggy




Sydney hasn't seen any blue sky like that for weeks.

BR

[JimmXinu]

It boggles my mind that a <1m DB file could cause physical disk access at all past the initial read. Surely it all fits in disk cache??

EDIT: Unless there's something that's cause write/flushes to occur?

[chaley]

Originally Posted by JimmXinu

It boggles my mind that a <1m DB file could cause physical disk access at all past the initial read. Surely it all fits in disk cache??

EDIT: Unless there's something that's cause write/flushes to occur?

I agree. It "smells" like there is something that touches the disk in some other way that isn't cached, such as fetching file attributes. Another thought is the same as yours: something is telling the OS to flush the cache, perhaps as part of a transaction commit.

[jnikkir]

Originally Posted by BetterRed

But here's the rub, VM switching on jnikkir's library was also rattly - I could hear disk r/w heads moving - that's not happening on my bigger, and arguably more complex libraries.

I can't figure out why this tiddly little library should be rattling the disk drives cage but there's no doubt it is.

I can hear that too! I wasn't sure whether that was normal or even how to describe it, but mine rattles too when I try to switch views, and when it stops rattling, the view finally loads.

Originally Posted by ownedbycats

I'd be curious to see what FileActivityWatch reports.

What am I looking for in this report? It's spitting out a lot of data -- is there something specific I should look for?

Originally Posted by chaley

If you send both the library and the calibre configuration folder (zipped up) to me then I will take a look. However, all I have is an SSD so given BetterRed's experience I might not see the problem.

Have you tried running Library / Library Maintenance / Check library? It is possible that the database is very fragmented.

Originally Posted by BetterRed

I did a Library Check - made no difference.

Still quite laggy on a rather old Flash thumb drive - no sound effects though

BR

Yeah I did a Check Library and it seems fine.

Sending those files to Chaley!

[chaley]

Originally Posted by jnikkir

Sending those files to Chaley!

I think I found what is happening but I have no idea why. Action Chains is calling gui.save_state many times each VL switch. Each call writes to the database, and might trigger a .25 second delay in Action Chains.

Here is a log that shows the calls. I added a print('save state XXX') that appears each time it is called. Someone else (@capink?) must look at why it is being called so often.

FWIW: I think that the view manager setup should include the VL for each of the views so that clicking the tab and selecting the view in the View Manager do the same thing,

[jnikkir]

Originally Posted by chaley

I think I found what is happening but I have no idea why. Action Chains is calling gui.save_state many times each VL switch. Each call writes to the database, and might trigger a .25 second delay in Action Chains.

Here is a log that shows the calls. I added a print('save state XXX') that appears each time it is called. Someone else (@capink?) must look at why it is being called so often.

FWIW: I think that the view manager setup should include the VL for each of the views so that clicking the tab and selecting the view in the View Manager do the same thing,

Oh man, finally some headway!! Thank you so much! I added that setting to each view in my VM setup. Should I jump over to the Action Chains plugin thread and ask capink about the save_state thing there?


Edit: Now that I think about it, does this account for the lag even when Action Chains wasn't doing anything, though? Just manually changing views without clicking on VL tabs has a lag too...

[chaley]

Originally Posted by jnikkir

Should I jump over to the Action Chains plugin thread and ask capink about the save_state thing there?

I just did it.