Two suggestions for Full text search

[albell]

Hi Kovid,

I'm curious to know if you plan to improve the full text search results window sometime in the future.
The whole feature is simply great, but at the moment the most uncomfortable thing to me is the order in which the matched books are displayed: I would not be wrong, but it seems completely random to me, or maybe they follow the date of insertion in Calibre?
Also, unless I have missed something, it is not possible to sort the results in any way. It would be great if matched books were sortable according to various parameters (at least the basic ones: author, tag, date, etc.),
and maybe show them in cover mode, just like in the Powersearch plugin.

Another thing that puzzles me is that if you open a book from the full text search window, the viewer by default searches for the snippet shown in the window. Which is useful, but I would find it more useful that the viewer opened searching the text with the initial query, so that you can immediately see if there are other results besides the first one shown by default by FTS.
I don't know however if it is possible to replicate some types of queries on the viewer (for example the one with NEAR), I've never tried.

Thank you!

[kovidgoyal]

results are sorted by relevance, aka, best match to your query. This is how pretty much all full text searches work. No, I have no plans to implement a cover based view, when searching for text seeing context around the match is far more important. If you want to browse books by some other criteria than relevance, use the mark all button and then switch to the main calibre window, do a search for marked:true and then sort the resulting list however you like.

The viewer cannot search for the original query, since the viewer does not support the same search syntax as the FTS engine.

[albell]

Originally Posted by kovidgoyal

If you want to browse books by some other criteria than relevance, use the mark all button and then switch to the main calibre window, do a search for marked:true and then sort the resulting list however you like.

Thanks, that works perfectly.

Originally Posted by kovidgoyal

The viewer cannot search for the original query, since the viewer does not support the same search syntax as the FTS engine.

Yes, that's what I feared.
So in the case of a query with NEAR, having only one occurrence shown in the FTS windows is a bit limiting, since it cannot be replicated on the viewer.
Just out of curiosity: is the choice to show only one result due to performance reasons or other? Will it be tweakable in the future, at least for queries that cannot be replicated on the viewer?

Thank you!

[kovidgoyal]

That's just how the FTS engine works (indeed most fts engines work). Think of for example a google search. They dont return multiple matches per webpage.

As for replicating near queries, you can do so in the viewer, using regular expressions. So for example to find the words one and two near each other you can search for

(?s)\bone\b.{3,150}\btwo\b

The only caveat is that the numbers above are characters not words, so its not quite as precise as in the FTS search.

[albell]

Originally Posted by kovidgoyal

That's just how the FTS engine works (indeed most fts engines work). Think of for example a google search. They dont return multiple matches per webpage.

As for replicating near queries, you can do so in the viewer, using regular expressions. So for example to find the words one and two near each other you can search for

(?s)\bone\b.{3,150}\btwo\b

The only caveat is that the numbers above are characters not words, so its not quite as precise as in the FTS search.

Thank you, this is a great workaround. Another caveat, I think, is that with regular expression the words are only searched in the order established in the expression, but that's not a big deal.

BTW I have another question:

I noticed that if I perform ocr on a scanned pdf that is already in the library (by launching an external app such as Acrobat), and then I save it overwriting the previous one, this is not automatically indexed and in fact remains "invisible" to FTS engine.
Is there a way to force indexing of a book (other than delete it and re-import it into the library)?

Thank you!

[ownedbycats]

Originally Posted by albell

BTW I have another question:

I noticed that if I perform ocr on a scanned pdf that is already in the library (by launching an external app such as Acrobat), and then I save it overwriting the previous one, this is not automatically indexed and in fact remains "invisible" to FTS engine.
Is there a way to force indexing of a book (other than delete it and re-import it into the library)?

Thank you!

Right-click in the details pane (or window) and there's an option to re-index.

In the future, copying the PDF file out and then re-adding it via the MDE will trigger the re-index and update the listed filesize.

[albell]

Originally Posted by ownedbycats

Right-click in the details pane (or window) and there's an option to re-index.

In the future, copying the PDF file out and then re-adding it via the MDE will trigger the re-index and update the listed filesize.

thanks, I didn't notice that!

[albell]

Originally Posted by ownedbycats

Right-click in the details pane (or window) and there's an option to re-index.

In the future, copying the PDF file out and then re-adding it via the MDE will trigger the re-index and update the listed filesize.

I did some testing and at the moment this last option is already the only one that works: remove the pdf from the MDE and add it again.
While if you reindex from the book's details pane, it still remains unindexed.

[kovidgoyal]

Originally Posted by albell

While if you reindex from the book's details pane, it still remains unindexed.

That is a bug fixed in the next release. You click on the PDF link in the book details panel and choose re-index and it will work.