Quote:
Originally Posted by kiwidude
I did find one other performance bottleneck which is the set_marked_ids when it is called with 50k rows (taking 11 seconds on my machine). I was progressively adding print statements through calibre source to find out where the slowdown is, got as far as refresh_ids() being invoked by marked_changed().
So my plugins normally do these two pairs of calls, for example:
Code:
self.gui.current_db.set_marked_ids(marked_ids)
self.gui.search.set_search_string('marked:library_duplicate')
|
The refresh_ids() is necessary to update the marked pins in the gui. It shouldn't be too slow. For example, in one library with 2500 books, selecting all the books then using "mark with text", the response is instantaneous.
Question: are you calling set_marked_ids in a loop with the dict growing each time, or once with a complete dict? If the former then that might explain it because all the work is redone for each call. If the latter then
Another test: how long does it take if you select all 50,000 books in the library then use Mark books to set a text mark? If this takes 11 seconds then we know the culprit. If it doesn't then there is something in the sequence of calls.
Quote:
|
In this situation because I am issuing a search straight after calling set_marked_ids I wonder if there is some expensive part of this "refresh" going on that doesn't actually need to happen because of the search afterwards. Or maybe it is that refresh which makes the search so fast of course...
|
The search will use the marked dict to get the list of IDs, a very fast operation. It does depend on the marks being set, but it doesn't depend on the marks being refreshed in the GUI.
If you are willing, could you give me the metadata.db for the 50,000 book library? With that I could look at what it actually happening. It may be that we could add a "refresh=True" parameter to set_marked_ids(), telling it whether or not to do the actual refresh. My theory: if you do the search at some point after then the refresh_ids will occur there, involving only the books returned by the search.