Slow to write tags to disk?

abrogard · 03-09-2025, 08:01 PM

Seems to me this library I'm working on has suddenly gotten slow to write the changes to disk when I add tags.

It is a 2000 book library. The disk has plenty of space. It's win10.

Task Manager reports no problems with any of space, cpu or memory.

Anything I could learn about this or should know? Tests I could do? Configs I should look at?

Karellen · 03-09-2025, 08:24 PM

How many ebooks at a time are you updating the tags. A few or hundreds?
How slow is "slow"?
When you update the tags or any info in the metadata screen, the database is updated and so is the metadata.opf file next to the ebook file.
So if you update 2000 books with a new tag, then 2000 metadata.opf files need to be rewritten also.
I don't know if that can be disabled... I've never needed to go looking for such a setting.

abrogard · 03-09-2025, 08:35 PM

I update one at a time. ("edit metadata individually")

How slow? Press 'OK' and count: "One and Two and Three.............." generally get up to Eight or Nine.

updating a database need not be a lengthy business. you simple write a record. the software selects that record and writes it. it's not like it has to rewrite all the other records. doesn't work like that.

So I doubt it's that.

And the computer is doing reads from the disk quick enough. And from Calibre.

I can double click on a book and it opens fairly smartly, whatever the format. Then I can scroll down to the end of the book very rapidly, no problem, check its all there, all good.

It is just this writing of the tag string.

theducks · 03-09-2025, 08:36 PM

Metadata.opf is a background task. You can even quit Calibre and it will resume the next time (if not completed). Its status is the first item on the Library Maintenance menu

BetterRed · 03-09-2025, 09:02 PM

I got a voice call interrupt - but I'll leave this here anyway

The metadata.opf files are written in a separate background thread…known as 'lazy writes'. Last time I looked it they were written at intervals of about 2 secs. If calibre is closed whilst there are outstanding metadata.opf writes the process will continue next time calibre is started.

@abrogard - the Library maintenance->Library metadata backup status action will report how many are outstanding.

The calibredb backup_metadata command can be used to flush outstanding lazy-writes of metadata.opf files. I include it in my daily backup process.

Library closed plugin to flush any outstanding lazy writes of metadata.opf files for all known libraries.

BR

BetterRed · 03-09-2025, 09:15 PM

@abrogard - Calibre version? Operating system and version?

What about other columns, e.g. Publisher, Rating, Comments etc are they slow too.

Run with debug, do an edit and post results here - there's an option in the Preferences drop down.

Run with no plugins, there's an option in the Preferences drop down.

BR

abrogard · 03-09-2025, 09:18 PM

Quote:

Originally Posted by BetterRed

I got a voice call interrupt - but I'll leave this here anyway

The metadata.opf files are written in a separate background thread…known as 'lazy writes'. Last time I looked it they were written at intervals of about 2 secs. If calibre is closed whilst there are outstanding metadata.opf writes the process will continue next time calibre is started.

@abrogard - the Library maintenance->Library metadata backup status action will report how many are outstanding.

The calibredb backup_metadata command can be used to flush outstanding lazy-writes of metadata.opf files. I include it in my daily backup process.

Library closed plugin to flush any outstanding lazy writes of metadata.opf files for all known libraries.

BR

Well that's very interesting. Perhaps explains why the metadata screen reports 'not responding' whilst I'm waiting.
I wondered what was not responding and surmised it must be some function being called. So that's it. Fine.

But it would seem to suggest I can stack up a queue of updates though, and I can't see that I can. Be fine if I could: I'd just keep going on editing and let it all sort itself out later... good... just what computers are for...

I cannot continue with my editing until that metadata update screen clears.

BetterRed · 03-09-2025, 09:31 PM

Something going on with your system, I just added and deleted a couple of tags, pressed OK and the MDE dialogue was gone instantly… no delay.

How about answering my questions, running debug and posting the log, and trying without plugins.

Windows 10, calibre 7.25

BR

BetterRed · 03-09-2025, 09:42 PM

And I just did a bulk change to a tag like column to 97 books in my Test library - again when I pressed OK the bulk MDE was gone instantly. Here's a screen shot of the Library metadata backup status :

Click image for larger version

Name: Screenshot 2025-03-10 123749.jpg
Views: 34
Size: 18.4 KB
ID: 214234

BR

theducks · 03-09-2025, 10:04 PM

You might try and exclude the Library folder from your A/V Active scan (only).
The book files should have been scanned when you downloaded/imported them.
If you have a really big metadata.db, it is possible that is the issue.
Mine is 60MB and a bulk metadata change is almost instant.

What drive (type and location)?

abrogard · 03-09-2025, 10:24 PM

Quote:

Originally Posted by BetterRed

Something going on with your system, I just added and deleted a couple of tags, pressed OK and the MDE dialogue was gone instantly… no delay.

How about answering my questions, running debug and posting the log, and trying without plugins.

Windows 10, calibre 7.25

BR

steady, steady, I'm getting there... bit by bit..

here's the debug log of a standard startup. I'll now do a startup without plugins - and debug - if I can..

Spoiler:

abrogard · 03-09-2025, 10:28 PM

Quote:

Originally Posted by BetterRed

@abrogard - Calibre version? Operating system and version?

What about other columns, e.g. Publisher, Rating, Comments etc are they slow too.

Run with debug, do an edit and post results here - there's an option in the Preferences drop down.

Run with no plugins, there's an option in the Preferences drop down.

BR

I ran without plugins and tried. It was just as tardy writing metadata.

here's the debug log from that:

Spoiler:

kovidgoyal · 03-09-2025, 10:32 PM

Try rebooting in safe mode and see if the performance improves.

abrogard · 03-09-2025, 10:38 PM

Quote:

Originally Posted by theducks

You might try and exclude the Library folder from your A/V Active scan (only).
The book files should have been scanned when you downloaded/imported them.
If you have a really big metadata.db, it is possible that is the issue.
Mine is 60MB and a bulk metadata change is almost instant.

What drive (type and location)?

Sata 1T drive internal. It is the system drive: C:

db is ony 3.5MB.

abrogard · 03-09-2025, 11:03 PM

Quote:

Originally Posted by kovidgoyal

Try rebooting in safe mode and see if the performance improves.

Thanks, Kovid. Tried it and it did not improve.

Tried the other suggestion, too: run without plugins - didn't help either.

And it is the same with other fields, such as 'comments'.

What about this suggestion of excluding something from 'active scan' ?

Maybe I should reinstall calibre might be a good first step?

03-09-2025, 08:01 PM	#1
abrogard Groupie Posts: 168 Karma: 10 Join Date: May 2017 Device: kindle	Slow to write tags to disk? Seems to me this library I'm working on has suddenly gotten slow to write the changes to disk when I add tags. It is a 2000 book library. The disk has plenty of space. It's win10. Task Manager reports no problems with any of space, cpu or memory. Anything I could learn about this or should know? Tests I could do? Configs I should look at?

03-09-2025, 09:02 PM	#5
BetterRed null operator (he/him) Posts: 21,485 Karma: 29308976 Join Date: Mar 2012 Location: Sydney Australia Device: none	I got a voice call interrupt - but I'll leave this here anyway The metadata.opf files are written in a separate background thread…known as 'lazy writes'. Last time I looked it they were written at intervals of about 2 secs. If calibre is closed whilst there are outstanding metadata.opf writes the process will continue next time calibre is started. @abrogard - the Library maintenance->Library metadata backup status action will report how many are outstanding. The calibredb backup_metadata command can be used to flush outstanding lazy-writes of metadata.opf files. I include it in my daily backup process. Library closed plugin to flush any outstanding lazy writes of metadata.opf files for all known libraries. BR Last edited by BetterRed; 03-09-2025 at 09:06 PM.

03-09-2025, 09:42 PM	#9
BetterRed null operator (he/him) Posts: 21,485 Karma: 29308976 Join Date: Mar 2012 Location: Sydney Australia Device: none	And I just did a bulk change to a tag like column to 97 books in my Test library - again when I pressed OK the bulk MDE was gone instantly. Here's a screen shot of the Library metadata backup status : BR

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Tags not stored when saving to disk?	Phssthpok	Library Management	14	11-09-2018 01:48 AM
Saving to disk doesn't write all the metadata into the PDF	1gnition	Library Management	3	08-15-2013 04:28 AM
How did kobo write such slow software?	notzed	Kobo Developer's Corner	4	01-15-2013 11:12 PM
Save to disk and Tags	tselling	Calibre	2	08-30-2009 09:05 PM
Write Fast, Write Slow?	Moejoe	Writers' Corner	14	03-25-2009 09:55 AM

03-09-2025, 08:24 PM	#2
Karellen Wizard Posts: 1,543 Karma: 9250680 Join Date: Sep 2021 Location: Australia Device: Kobo Libra 2	How many ebooks at a time are you updating the tags. A few or hundreds? How slow is "slow"? When you update the tags or any info in the metadata screen, the database is updated and so is the metadata.opf file next to the ebook file. So if you update 2000 books with a new tag, then 2000 metadata.opf files need to be rewritten also. I don't know if that can be disabled... I've never needed to go looking for such a setting.

03-09-2025, 08:35 PM	#3
abrogard Groupie Posts: 168 Karma: 10 Join Date: May 2017 Device: kindle	I update one at a time. ("edit metadata individually") How slow? Press 'OK' and count: "One and Two and Three.............." generally get up to Eight or Nine. updating a database need not be a lengthy business. you simple write a record. the software selects that record and writes it. it's not like it has to rewrite all the other records. doesn't work like that. So I doubt it's that. And the computer is doing reads from the disk quick enough. And from Calibre. I can double click on a book and it opens fairly smartly, whatever the format. Then I can scroll down to the end of the book very rapidly, no problem, check its all there, all good. It is just this writing of the tag string.

03-09-2025, 08:36 PM	#4
theducks Well trained by Cats Posts: 30,755 Karma: 59473090 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	Metadata.opf is a background task. You can even quit Calibre and it will resume the next time (if not completed). Its status is the first item on the Library Maintenance menu

03-09-2025, 09:15 PM	#6
BetterRed null operator (he/him) Posts: 21,485 Karma: 29308976 Join Date: Mar 2012 Location: Sydney Australia Device: none	@abrogard - Calibre version? Operating system and version? What about other columns, e.g. Publisher, Rating, Comments etc are they slow too. Run with debug, do an edit and post results here - there's an option in the Preferences drop down. Run with no plugins, there's an option in the Preferences drop down. BR

03-09-2025, 09:31 PM	#8
BetterRed null operator (he/him) Posts: 21,485 Karma: 29308976 Join Date: Mar 2012 Location: Sydney Australia Device: none	Something going on with your system, I just added and deleted a couple of tags, pressed OK and the MDE dialogue was gone instantly… no delay. How about answering my questions, running debug and posting the log, and trying without plugins. Windows 10, calibre 7.25 BR

03-09-2025, 10:04 PM	#10
theducks Well trained by Cats Posts: 30,755 Karma: 59473090 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	You might try and exclude the Library folder from your A/V Active scan (only). The book files should have been scanned when you downloaded/imported them. If you have a really big metadata.db, it is possible that is the issue. Mine is 60MB and a bulk metadata change is almost instant. What drive (type and location)?

03-09-2025, 10:32 PM	#13
kovidgoyal creator of calibre Posts: 44,955 Karma: 26594922 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Try rebooting in safe mode and see if the performance improves.

Advert

Advert