calibre 6 beta: Full text searching - Page 3

BetterRed · 06-27-2022, 07:47 PM

Quote:

Originally Posted by kovidgoyal

@BR: I dont see a good solution for using VLs and not using restrict books. I dont want to have calibre cancel an applied vl.

Understood and I agree… but how about having the software check the Restrict searched books if a VL or metadata search is active… if the user really wants to do a library wide text search whilst a metadata search is active they can uncheck it easy enough.

Quote:

Originally Posted by kovidgoyal

As for not showing the snippets, I dont really see the point, it doesnt take much in the way of resources to search through text (the text is cached the format files are not re-opened).

Four minutes to display the snippets for 250 books with nothing much else running seemed quite slow to me. It was using up to 25% of an i7-7700 with 16GB of RAM and a fair bit of HDD activity - that's why I thought it must be rattling through the format files. The full-text-search.db file is 3.51GB.

BR

ownedbycats · 06-27-2022, 08:09 PM

Question: Where will be the location of the search index? AppData or the library folder? I haven't tested yet to check.

BetterRed · 06-27-2022, 08:14 PM

Quote:

Originally Posted by ownedbycats

Question: Where will be the location of the search index? AppData or the library folder? I haven't tested yet to check.

Library folder - full-text-search.db

BR

kovidgoyal · 06-27-2022, 09:20 PM

Quote:

Originally Posted by Uncle Robin

A question before I install the beta - will it update in the usual fashion when 6 comes out of beta or will an uninstall/reinstall be required?

Yes, it will update normally.

kovidgoyal · 06-27-2022, 09:24 PM

Quote:

Originally Posted by BetterRed

Understood and I agree… but how about having the software check the Restrict searched books if a VL or metadata search is active… if the user really wants to do a library wide text search whilst a metadata search is active they can uncheck it easy enough.

But then that would mean the users choice for the checkbox is not remembered. Not sure that's better. I can think of plenty of use cases for searching full library even when the main screen is restricted. For example, to check if there are books with mislabelled data.

Quote:

Four minutes to display the snippets for 250 books with nothing much else running seemed quite slow to me. It was using up to 25% of an i7-7700 with 16GB of RAM and a fair bit of HDD activity - that's why I thought it must be rattling through the format files. The full-text-search.db file is 3.51GB.

BR

The snippet search has to go through the full book text, sadly the sqlite fts engine doesnt seem store offsets of words in the text, which is why it is slow. But it only needs to search the matched book files, not the full library.

ownedbycats · 06-27-2022, 09:55 PM

Can Calibre 6 install alongside 5, or on top of it? The uninstall-to-downgrade cycle is a bit of a pain for testing.

BetterRed · 06-27-2022, 10:07 PM

Quote:

Originally Posted by kovidgoyal

But then that would mean the users choice for the checkbox is not remembered. Not sure that's better. I can think of plenty of use cases for searching full library even when the main screen is restricted. For example, to check if there are books with mislabelled data.

The snippet search has to go through the full book text, sadly the sqlite fts engine doesnt seem store offsets of words in the text, which is why it is slow. But it only needs to search the matched book files, not the full library.

Fair 'nuff, helps to know what's going on under the covers.

Back to hidden windows. An undocked Quickview window stays on top if I give focus to the library list window, viz:

Click image for larger version

Name: Screenshot 2022-06-28 114923.jpg
Views: 704
Size: 394.0 KB
ID: 194607

I appreciate QV is a child window and TS is separate process (I can't Alt+Tab into QV)… but that may not be as obvious to some people.

BR

kovidgoyal · 06-27-2022, 10:21 PM

FTS and QV are both child windows they arent separate processes.

ownedbycats · 06-27-2022, 10:27 PM

I installed the beta. It switched to light mode. I guess it's following system theme rather than the env var. Disappointing.

Any plans to make an override in look & feel? I seem to recall mention of possibly implementing it, but I can't find the post now.

EDIT: This is going to take a while...

Click image for larger version

Name: 2022-06-27 23_31_40-Search the text of all books in the library.png
Views: 1060
Size: 21.4 KB
ID: 194608

ANOTHER EDIT: I feel like it would make sense to be able to dedicate xx CPU cores to indexing, similar to the option in preferences > miscellaneous.

DNSB · 06-27-2022, 11:00 PM

Quote:

Originally Posted by ownedbycats

Can Calibre 6 install alongside 5, or on top of it? The uninstall-to-downgrade cycle is a bit of a pain for testing.

I install the betas as portable versions. When the release arrives, I do a normal install.

BetterRed · 06-27-2022, 11:25 PM

Quote:

Originally Posted by kovidgoyal

FTS and QV are both child windows they arent separate processes.

Perhaps 'process' is the wrong term, but if they are both child windows, they have rather different behaviours… just like real children

1. QV doesn't provide a separate taskbar flyout item, TS does :

Click image for larger version

Name: Screenshot 2022-06-28 124238.jpg
Views: 338
Size: 466.7 KB
ID: 194609

2. QV is not available as a selectable item when I press Win(or Alt)+Tab, whereas TS is:

Click image for larger version

Name: Screenshot 2022-06-28 125823.jpg
Views: 274
Size: 230.0 KB
ID: 194610

3. If I minimise the main window, it takes QV with it, but it leaves TS (which cannot be minimised) where it is:

Click image for larger version

Name: Screenshot 2022-06-28 131612.jpg
Views: 272
Size: 669.8 KB
ID: 194611

4. When I give focus to the main window, QV stays on top, whereas TS gets pushed down. Why can't TS behave like QV in this regard?

BR

BetterRed · 06-27-2022, 11:26 PM

Quote:

Originally Posted by DNSB

I install the betas as portable versions. When the release arrives, I do a normal install.

BetterRed · 06-27-2022, 11:35 PM

Quote:

Originally Posted by ownedbycats

I installed the beta. It switched to light mode. I guess it's following system theme rather than the env var. Disappointing.

Any plans to make an override in look & feel? I seem to recall mention of possibly implementing it, but I can't find the post now.

EDIT: This is going to take a while...

Attachment 194608

ANOTHER EDIT: I feel like it would make sense to be able to dedicate xx CPU cores to indexing, similar to the option in preferences > miscellaneous.

Did you try pressing the Fast button, my Books library indexed ~6,900 files in a bit over an hour whilst I had dinner.

BR

ownedbycats · 06-27-2022, 11:36 PM

Yeah, the FTS window doesn't follow the behaviour of most other Calibre child windows I've tried (preferences, annotations, jobs, undocked quickview).

kovidgoyal · 06-27-2022, 11:47 PM

Yes the FTS window is *independent* of the main calibre window, you are supposed to be able to use both simultaneously. That's by design. @BR: When restoring the main window one cannot restore the FTS window since doing so would mean on a small screen the FTS window would always cover the main window.

That said, the I will look into adding minimize/maximize buttons to the FTS window titlebar. SHould be a simple flag to pass to Qt.

06-27-2022, 09:55 PM	#36
ownedbycats Custom User Title Posts: 11,000 Karma: 75337983 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	Can Calibre 6 install alongside 5, or on top of it? The uninstall-to-downgrade cycle is a bit of a pain for testing. Last edited by ownedbycats; 06-27-2022 at 10:04 PM.

06-27-2022, 10:27 PM	#39
ownedbycats Custom User Title Posts: 11,000 Karma: 75337983 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	I installed the beta. It switched to light mode. I guess it's following system theme rather than the env var. Disappointing. Any plans to make an override in look & feel? I seem to recall mention of possibly implementing it, but I can't find the post now. EDIT: This is going to take a while... ANOTHER EDIT: I feel like it would make sense to be able to dedicate xx CPU cores to indexing, similar to the option in preferences > miscellaneous. Last edited by ownedbycats; 06-27-2022 at 10:58 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Searching for Text in Footnotes	aherinandrianina	Calibre	1	12-05-2018 04:08 AM
Troubles searching text in files	Maxwell61	Library Management	9	09-18-2018 06:13 PM
searching google from e-reader text	garrry	Related Tools	2	06-15-2013 04:39 AM
searching a document based on highlighted text	KasPaz	Kindle Developer's Corner	0	02-06-2012 10:37 PM
Searching text on 300 and 505	Thrasher	Sony Reader	6	04-28-2010 03:41 AM

06-27-2022, 08:09 PM	#32
ownedbycats Custom User Title Posts: 11,000 Karma: 75337983 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	Question: Where will be the location of the search index? AppData or the library folder? I haven't tested yet to check.

06-27-2022, 10:21 PM	#38
kovidgoyal creator of calibre Posts: 45,377 Karma: 27230406 Join Date: Oct 2006 Location: Mumbai, India Device: Various	FTS and QV are both child windows they arent separate processes.

06-27-2022, 11:36 PM	#44
ownedbycats Custom User Title Posts: 11,000 Karma: 75337983 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	Yeah, the FTS window doesn't follow the behaviour of most other Calibre child windows I've tried (preferences, annotations, jobs, undocked quickview).

06-27-2022, 11:47 PM	#45
kovidgoyal creator of calibre Posts: 45,377 Karma: 27230406 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Yes the FTS window is independent of the main calibre window, you are supposed to be able to use both simultaneously. That's by design. @BR: When restoring the main window one cannot restore the FTS window since doing so would mean on a small screen the FTS window would always cover the main window. That said, the I will look into adding minimize/maximize buttons to the FTS window titlebar. SHould be a simple flag to pass to Qt.