Slow to write tags to disk?

[abrogard]

How about this? I find with windows process monitor that my calibre process is making over 2 million disk writes to update a few tags on one book. author, title, 'tags'.
Is that to be expected?

and I notice that the monitor reports a different 'write' operation for each 4 bytes. Like 20 consecutive bytes written would require 5 write operations. that surely is wrong? and what module is responsible for that? on the windows end or on the calibre end?

[kovidgoyal]

Originally Posted by abrogard

How about this? I find with windows process monitor that my calibre process is making over 2 million disk writes to update a few tags on one book. author, title, 'tags'.
Is that to be expected?

Definitely not. It does no such thing. You definitely have some software on your machine interfering with calibre.

[abrogard]

Originally Posted by kovidgoyal

Definitely not. It does no such thing. You definitely have some software on your machine interfering with calibre.

Well windows process monitor thinks it does. Sees it as such. If you can suggest how I look further I will.

I'v zipped up a segment of the monitor output in libre calc format and attached it.

[kovidgoyal]

That just means there is something on your machine inserting code into the calibre process. Sadly the windows ecosystem is full of such junk.

[abrogard]

You're saying it is a corrupt calibre install or you're saying that as calibre.exe runs the write functions are intercepted or such?
Any ideas on how to deal with it?
I never heard of 'inserting code' into a running process... how's that done? and how does it get found out?
You didn't look at the logfile did you?

[BetterRed]

Can you post a list of all the processes that are running - e.g. from ProcessExplorer

BR

[kovidgoyal]

Originally Posted by abrogard

You're saying it is a corrupt calibre install or you're saying that as calibre.exe runs the write functions are intercepted or such?
Any ideas on how to deal with it?
I never heard of 'inserting code' into a running process... how's that done? and how does it get found out?
You didn't look at the logfile did you?

It's how every antivirus with "live protection" on windows operates, for instance. Every write made by a processes gets intercepted and scanned for safety before being allowed to hit the filesystem. Similarly network operations are intercepted, etc etc.

No I didnt look at the log file, it's pointless I know perfectly well that calibre does not make 2 million writes when updating metadata for a single book. So they cant be coming from calibre code, a metadata update is literally updating a few rows in a sqlite database, proportional to the number of columns/fields you have in your library. And then there will be an eventual backup which is another single write to an OPF file.

[abrogard]

Originally Posted by kovidgoyal

It's how every antivirus with "live protection" on windows operates, for instance. Every write made by a processes gets intercepted and scanned for safety before being allowed to hit the filesystem. Similarly network operations are intercepted, etc etc.

No I didnt look at the log file, it's pointless I know perfectly well that calibre does not make 2 million writes when updating metadata for a single book. So they cant be coming from calibre code, a metadata update is literally updating a few rows in a sqlite database, proportional to the number of columns/fields you have in your library. And then there will be an eventual backup which is another single write to an OPF file.

ok, you call that 'inserting code'. alright. i don't see it as any insertion but have it your own way. the point is to reach clarity of understanding.
and in that direction my underlying point was that perhaps there would be something in the output that might be of interest and give some leads as to where to look.

which I think is a perfectly reasonable attitude. And if you don't then that's okay and just adds information about yourself.

I appreciate your help and interest.
I appreciate anyone's help and interest.

[abrogard]

Originally Posted by BetterRed

Can you post a list of all the processes that are running - e.g. from ProcessExplorer

BR

Yep. I've attached the output.

[theducks]

I see a Google drive process running.
Calibre (the Library folder) is NOT compatible with Google drive.
Make 100% sure the Library folder is located elsewhere (lot a GD monitores folder)

[BetterRed]

I looked at the log, it's the updating of the SQLite database that chewing up the time, the metadata.opf activity is diddlysquat. What appears to be temporary file I/O in the root of the config directory surprised me… calibre normally does such things in its cache directory.

Re Process Explorer. Nothing leapt off the page, just a couple of "I wonder what that is, and why is he running that. "

Can you try without running calibre when Firefox is not running. I recently discovered Firefox was making my system unstable… such as unwanted reboots… disabling DuckDuckGo Privacy Essentials seems to have fixed it.

Seeing as how the problem manifests in Windows Safe Mode, could you run calibre under it again and capture another ProcExplorer list.

BR

[abrogard]

Originally Posted by theducks

I see a Google drive process running.
Calibre (the Library folder) is NOT compatible with Google drive.
Make 100% sure the Library folder is located elsewhere (lot a GD monitores folder)

I shun google drive. I'm surprised it is there. I'll do what I can to remove it.

the library is on the actual hard drive, quite for sure. I moved it there. copied it over via usb stick, ' sneakernet'.

[abrogard]

Originally Posted by BetterRed

I looked at the log, it's the updating of the SQLite database that chewing up the time, the metadata.opf activity is diddlysquat. What appears to be temporary file I/O in the root of the config directory surprised me… calibre normally does such things in its cache directory.

Re Process Explorer. Nothing leapt off the page, just a couple of "I wonder what that is, and why is he running that. "

Can you try without running calibre when Firefox is not running. I recently discovered Firefox was making my system unstable… such as unwanted reboots… disabling DuckDuckGo Privacy Essentials seems to have fixed it.

Seeing as how the problem manifests in Windows Safe Mode, could you run calibre under it again and capture another ProcExplorer list.

BR

You will have noted the 4byte writes?

I'm busy writing a copy to usb again. I intend to move it to another machine and see if the problem persists there.

I will do the safe mode reboot and try when that copy finishes - takes forever.

you mean run without firefox running I think? I've tried that. didn't help.

[abrogard]

Here we are. Safe mode. No Firefox. Still takes forever.
And here is the process explorer output.
I did get the library onto another machine and ran it there.
No trouble at all.
So without doubt it is a Windows problem or, we could say I suppose, a possible windows/calibre conflict. There's not just me experiencing this.
Anyway here is the process explorer output for that session in safe mode.
It'll be lovely if it reveals something to someone somewhere, this is a real hassle.

[Karellen]

Originally Posted by abrogard

So without doubt it is a Windows problem or, we could say I suppose, a possible windows/calibre conflict.

It's not a general and widespread problem. Seems a problem on your system. I am a Windows 10 and now 11 user and haven't experienced the problem you describe.