why is calibre slow on batch operations?

[kovidgoyal]

You've gotta live with it. When many records have changed, resorting makes sense. You should be able to reduce the sort overhead by using a virtual library, though off hand I can't recall if the current database code restricts to the VL when sorting or not, I know the new db code does.

[kovidgoyal]

You can also reduce sort overhead by sorting ona simple column like date or a boolean/numeric column before bulk editing.

[kovidgoyal]

Actually on second thoughts, a virtual library probably wont help since after a bulk edit the VL itself has to be re-computed.

[mbovenka]

Originally Posted by kovidgoyal

Actually on second thoughts, a virtual library probably wont help since after a bulk edit the VL itself has to be re-computed.

Yes, my experience is that a virtual library doesn't speed it up. Oh well, a good excuse to get me a beefier box

[BetterRed]

Originally Posted by speakingtohe

Even with the tag browser off though it takes a long time to do a bulk metadata change of even one field in two records, so there is something else in play here.

Helen

To twenty books in a 24000+ book library I just :-

added four tags,
changed the publisher,
changed a custom date (last change date) and
changed a fixed value set custom column (status) which is shown in the tag browser

the spreadsheet was sorted on status (ascending) and last change date (descending), which is the default I have in View Manager, the tag browser was open with all categories collapsed, no virtual libraries or search terms were in effect, book details was displayed

Took < 7secs - I regard that as plenty good enough.

Sometimes I change the metadata on 100's of books, I can't recall any that were unacceptably slow, like more than few minutes. I rarely need to change author names or titles - I try to ensure they are correct before I move the books from my 'new books' library to my 'main' library.

I do not have any columns derived/calculated from other columns, nor do I have coloured columns, or other decorative effects, and I don't use the cover browser - i.e as vanilla as it comes.

My calibre system has an i5 processor with 6GB RAM. The library is on a WD 2TB internal spinning disk, on a SATA controller, its formatted for NTFS, and the drive is fully indexed for Windows search and Locate 32. I don't use an SSD, or RAM Disks or any of that tricky stuff. The database is in the library folder with the author & book folders and its 109MB.

The software and config data are on another 2TB WD drive.

BR

[mbovenka]

Originally Posted by BetterRed

:My calibre system has an i5 processor with 6GB RAM. The library is on a WD 2TB internal spinning disk, on a SATA controller, its formatted for NTFS, and the drive is fully indexed for Windows search and Locate 32. I don't use an SSD, or RAM Disks or any of that tricky stuff. The database is in the library folder with the author & book folders and its 109MB.

The software and config data are on another 2TB WD drive.

BR

See what I mean by 'beefier box'? . I'm talking an old laptop, a Core2Duo with 1.5GB RAM and a 71K books library on a USB2 external disk.

Starting up takes 3 minutes or so, but after that, it's surprisingly workable.

[theducks]

About 32 seconds full startup (Library screen stabilized)
the Logo appeared after ~11 seconds

A lowly P4 2.4GHz w 2G RAM
Calibre program on Main HD
Library on External 2.5" HD connected by USB2

Note: Starting up with Calibre Portable (located on that same USB2 HD only added 2 seconds to 'Logo' and the total

[chaley]

Originally Posted by kovidgoyal

You should be able to reduce the sort overhead by using a virtual library, though off hand I can't recall if the current database code restricts to the VL when sorting or not, I know the new db code does.

It does a search of all records to build the VL, then sorts the results of that search. One would need to see the VL search expression to know which of those two operations is the slower one.

[speakingtohe]

Originally Posted by mbovenka

Aha. Yeah, that would explain it. Is there a way around it, or is it just a question of 'Sorry, you gotta live with it'?

At one time there was no resort IIRC. I would rather not have it but many must prefer it. A tweak would be nice

Helen

[BetterRed]

Originally Posted by speakingtohe

At one time there was no resort IIRC. I would rather not have it but many must prefer it. A tweak would be nice

Helen

I too would like a no re-sort or re-select tweak, not because of performance, but because a bulk edit can take books from the current selection criteria and/or virtual library. Thankfully this doesn't happen if I edit a book metadata in the spreadsheet

I'd rather use the Go button as & when I want the spreadsheet refreshed.

BR

[mbovenka]

Originally Posted by BetterRed

I too would like a no re-sort or re-select tweak, not because of performance, but because a bulk edit can take books from the current selection criteria and/or virtual library. Thankfully this doesn't happen if I edit a book metadata in the spreadsheet

I'd rather use the Go button as & when I want the spreadsheet refreshed.

BR

Yeah, that would be another plus of no re-sorting.

[kovidgoyal]

https://github.com/kovidgoyal/calibr...7b070cf5ab97fa

[mbovenka]

Originally Posted by kovidgoyal

https://github.com/kovidgoyal/calibr...7b070cf5ab97fa

And Kovid comes through again Time for another donation...

[BetterRed]

Originally Posted by mbovenka

And Kovid comes through again Time for another donation...

Kovid

BR

[mbovenka]

Originally Posted by kovidgoyal

https://github.com/kovidgoyal/calibr...7b070cf5ab97fa

Just tried 0.9.39.