Quote:
Originally Posted by kiwidude
It is even slower using the GUI to mark books  . Clearing is super fast. Mark books with text label takes about the 11 seconds to appear in the dialog, then another 17 seconds to apply the mark. |
Thank you for the db. That helped enormously!
I found yet another 'hidden' linear search deep in the guts of the gui's view of the database. The code verified that a book_id exists in the gui display (has a gui row number) by searching for the book in a list. It being a list, the average number of "looks" to make the check is len(books)/2. It did this for every book_id in the list of ids passed to refresh_ids(). For your db with all 50,000 books selected that meant 1.25 billion lookups (!) to refresh the booklist. The fix is to keep a dict {book_id:row} so the check is one dict probe per book id.
I also found a place where on a mouse click calibre checked to see if the clicked-on row was in a selection, and if so checked some other things to see if drag-n-drop is being used. By refactoring the check I knocked 3 seconds off the right-click on a book.
Qt takes a long time to return the selected rows. I can't do anything about that.
The bottom line (all times measured on my machine, an Intel Core i7-10710U):
- It still takes around 4 seconds to select all 50,000 books with Ctrl-A. This time is all in Qt.
- Opening the context menu for a book is now instantaneous. It was around 4 seconds.
- It takes 4 seconds to open the mark with text dialog for the same reason it takes 4 seconds to select the books: Qt returning the selected rows.
- It takes 30 milliseconds after hitting OK to mark the books and refresh the gui. This took 12.2 seconds before the changes.
What took 24 seconds before now takes 8, and all of that 8 is in Qt selection management.
I will submit the changes to Kovid. The fix is a bit exotic so he might prefer an alternate approach. I will post here once we have a resolution in source.