Displaying arbitrary results in search screen from plugin

[kiwidude]

One of the plugin suggestions that came up was one that has the net effect of wanting to display rows in Calibre by some criteria that isn't handled by the search logic. Say for instance you wanted to find all books that had a cover smaller than certain dimensions/size.

So the plugin iterates through the data set and ends up with a bunch of calibre ids it wants to display in the library view.

Is there a way to display these? It's a not dissimilar issue to the discussion we had on duplicate detection - except for that we think adding a custom column is appropriate in which case a normal search based on that column is possible. Is adding a custom column the only way to display arbitrary ids?

[chaley]

What should work today is library_view.select_rows(). This is not a search, but instead will highlight the rows in the view in the same fashion as search w/highlight search results. The user can use next and previous to move around in the highlighted set. This technique will not interfere with searching and restrictions.

'Searching' for a collection of arbitrary IDs would be more complicated. The problem is that searching is done by restricting the content of the cache (tied in with restrictions), which the GUI then treats as the library. Changing this processing is almost certainly not going to happen. I could imagine a special search syntax, something like 'ids:n,n,n,n,n,n,n,n,n', but this wouldn't be good if there are 1000s of them.

[kiwidude]

Thanks Charles, I think it's time I tried that highlight feature you keep mentioning to see what it does for myself .

[kiwidude]

Originally Posted by chaley

What should work today is library_view.select_rows(). This is not a search, but instead will highlight the rows in the view in the same fashion as search w/highlight search results. The user can use next and previous to move around in the highlighted set. This technique will not interfere with searching and restrictions.

So I decided to give this approach a go. I have built up a list of ids I want to highlight, and have code something like this:

self.gui.search.clear() # As my code works across the whole library
invalid_ids = [... build up my model ids...]
self.gui.library_view.model().set_highlight_only(True)
self.gui.library_view.select_rows(invalid_ids)

However all that does is select the rows, it won't actually color them. The highlighting code seems to take place in the search code in BooksModel, where it sets those "ids_to_highlight" etc.

Any suggestions as to what else I need to do?

[kiwidude]

Ok, this is working, I am sure someone can slap me around if there is a better approach:

        model = self.gui.library_view.model()
        model.ids_to_highlight = invalid_ids
        model.ids_to_highlight_set = set(invalid_ids)
        model.current_highlighted_idx = 0
        self.gui.library_view.select_rows(invalid_ids[:1])

[chaley]

I am not happy about you needing to reach inside the model like that. I can provide an API to do the 'right' way. However, before I do that, is the highlighting solution adequate?

[kiwidude]

Thought you might want to wrap it up

In a perfect world I think I would still prefer a way of limiting the view to just the invalid ids, but I can appreciate as per your initial response this isn't something easily possible currently. The disadvantage of the highlighting approach is that it requires leaving the view in "highlight mode" while the user works through those rows, and then they need to perform an action to get back out of it and return to normal searching. I've got a keyboard shortcut in my plugin to toggle turning highlight mode on/off. However there is no visible indication in the main gui as to whether you are in highlight mode or not (you have to drill into the search preferences dialog) so that is a little icky. I think as a user I might have preferred the checkbox (or a toggle button) on next to the search box as it used to be as it least you could see what "mode" you currently are in.

With that all said, as you have the N/shift-N keyboard shortcuts to work through the results it functionally does the job. It just isn't quite as nice imho as limiting the set of rows to just the ones you are interested in rather than jumping all around the results, particularly since there is no way to sort to show the highlighted rows at the top.

[chaley]

Originally Posted by kiwidude

The disadvantage of the highlighting approach is that it requires leaving the view in "highlight mode" while the user works through those rows, and then they need to perform an action to get back out of it and return to normal searching.

I don't understand. I have a prototype in model() that sets the highlighted rows. They get highlighted. I am not required to turn highlight mode on or off. Are you sure that you need to change the toggle?

[chaley]

How many rows are we talking about here? Tens, or tens of thousands?

[chaley]

If you could try this before I submit to Kovid, that would be useful.

Add in gui2.library.models.py at line 280, just before 'search(...)

    def highlight_ids(self, ids_to_highlight):
        self.ids_to_highlight = ids_to_highlight
        self.ids_to_highlight_set = set(self.ids_to_highlight)
        if self.ids_to_highlight:
            self.current_highlighted_idx = 0
        else:
            self.current_highlighted_idx = None
        self.reset()

[kiwidude]

Originally Posted by chaley

I don't understand. I have a prototype in model() that sets the highlighted rows. They get highlighted. I am not required to turn highlight mode on or off. Are you sure that you need to change the toggle?

I didn't realise it was possible to highlight the rows without switching the highlight mode. I've just tried your code now and that works well - I just hit esc and my highlight mode state is preserved whether it was on or off.

In terms of rows, it could be thousands, it could be a handful. Really depends on the state of the users library

Perhaps it is possible to supply some custom sorting to the view with a delegate to display the highlighted rows at the top (in my case I just want to test for presence in the highlighted set)?

[chaley]

Originally Posted by kiwidude

Perhaps it is possible to supply some custom sorting to the view with a delegate to display the highlighted rows at the top (in my case I just want to test for presence in the highlighted set)?

This has a problem similar to search -- the model.py must be asked to do the work, somehow cause cache.py to order the records in the cache (using data in the cache), then redisplay the reordered cache.

It really sounds like you should create a (hidden?) custom column, fill that in how you want, then set a search restriction to select rows with appropriate values in the column. That would do everything you have described, including permitting sub-searching.

[kovidgoyal]

It may be worth using the (currently unused) flags column for this kind of thing. We could provide an API to set the value for it and then have a true false search on it. That way every use would not require a separate custom column.

[kovidgoyal]

In fact there's no need to use the actual column, we could create a virtual column in meta2. This would not persist between calibre restarts, but I see that as a plus.

[chaley]

Originally Posted by kovidgoyal

In fact there's no need to use the actual column, we could create a virtual column in meta2. This would not persist between calibre restarts, but I see that as a plus.

Hmmm...

That should work easily enough. I will give it a try. What column name would you like to see in field_metadata?