Calibre Internal Process While Updating

[Evilwarning]

Hi

Would it be possible to tell what are the steps that calibre does when we update a specific field. For Example if i update the cover or details, does it only update the database or does any sorting, etc ?

Thank you

[Starson17]

Originally Posted by Evilwarning

Hi

Would it be possible to tell what are the steps that calibre does when we update a specific field. For Example if i update the cover or details, does it only update the database or does any sorting, etc ?

Thank you

It varies, so you'll need to be more specific. When metadata is changed, the metadata.db SQL database is updated and the metadata.opf file is updated for that record. Updating the database may cause certain SQL triggers to fire, and that may generate sort fields, cause the records to be moved (if title or author change) etc.

[kovidgoyal]

depends on exactly how you are updating. A bulk update will resort, a in list update or a metadata single update will not.

[Evilwarning]

I have a large collection of books and every time i make a change to the metadata 10-15 seconds the user interface is unavailable.

to test i have loaded 500 books into a empty library and runs really fast and response, i assume that the system must not be just updating the database but must be doing some operations. Something besides updating the opf file and the database? otherwise would be as response in 20000 as is in 500.

Just talking on the case of updating one book at the time.

Same thing happens when merging book titles.

Thank for your help

[kovidgoyal]

how are you making the change?

[Evilwarning]

E for the details page and than updating ISBN or/and Cover. But i see the effect on changing any data.

I see the effect too when i just change the author details directly in the datagrid

[kovidgoyal]

strange, 15-20 secs seems excessive, open a ticket for it and I'll look into it when I have the time.

[chaley]

My guess is that the time is in tag_view.recount. It calls (indirectly) db.get_categories(), which takes time proportional to the number of tags/authors/publishers, etc. If a) there are lots of items, and b) not enough memory for sqlite to cache the DB, then get_categories will be slow.

@evilwarning: what sort of machine are you running calibre on? In particular, how much RAM do you have, and is your DB on a local disk or a NAS?

[Evilwarning]

Hi

I have a Mac Pro with 4 Cores and 4 GB of memory.

[Evilwarning]

and from local disk 1TB Western digital Green 32mb Cache

[chaley]

I don't see why with that configuration you should be seeing the times you do, unless you have zillions of tags and authors. That said, the next release has changes that will help. First, closing the tag pane will stop all the work needed to update it. Second, the time to update the tags pane has been reduced by a factor between 5 and 6 times.

We will see whether these helps you, or whether your machine is haunted.

[Evilwarning]

well, 20000 books, around 16000 tags. I'll try closing the tags pane

Thank you for your help

[Evilwarning]

Hi

Tested removing the Tag Browser, didn't have any effect.
I Upgraded to 0.7.34 to check the improvements you mentioned, now works FAST, but like FAST and i mean FAST, no delay on updating metadata, on merging. Just perfect!!!

Thank you very much for all your work and effort

[chaley]

Originally Posted by Evilwarning

Tested removing the Tag Browser, didn't have any effect.
I Upgraded to 0.7.34 to check the improvements you mentioned, now works FAST, but like FAST and i mean FAST, no delay on updating metadata, on merging. Just perfect!!!

Thank you very much for all your work and effort

You are welcome. Glad to hear it helped.

The tag browser open/closed change also appeared in 0.7.34, so closing it wouldn't have had any effect until you upgraded. After upgrade, you might be able to see the difference between the browser open and closed. In one of my large library tests, I could see around a 1/2 second difference, enough to be very noticeable but not enough to worry about. But as my library had 5,000 tags instead of 16,000, you might see more of a difference.