FTS: Book count etc.

[Quoth]

I've 6337 "books" in Calibre. Some might have no files. Some might be image based PDF (may or may not have OCR layer).
I started the indexing for FTS, but it's indexing 14249 books!

Where there are multiple formats can it be set to only index epub, which would always exist of more than one format?

What does it do with PDFs that have either text or an OCR text layer, or at all?

Can I exclude rows from being indexed at all?

[Quoth]

Now at 26% and ~2 hours 18 minutes estimate. Set to "Fast".

[Comfy.n]

Originally Posted by Quoth

Where there are multiple formats can it be set to only index epub, which would always exist of more than one format?

Indeed, this would be great to make the FTS database smaller and bypass the redundant results, not sure if it's possible... I think this question has been raised when the feature was implemented :\

[Comfy.n]

Originally Posted by Quoth

Now at 26% and ~2 hours 18 minutes estimate. Set to "Fast".

I set my initial indexing to run overnight

[kovidgoyal]

No it will index all formats, and you cannot exclude books. As for PDF it extracts teh text using the pdftotext tool.

[Quoth]

Originally Posted by kovidgoyal

No it will index all formats, and you cannot exclude books. As for PDF it extracts teh text using the pdftotext tool.

That's possibly handy for some of the PDFs.

Thanks.

So that explains indexing 14249 vs 6337 "books" in Calibre.

[Quoth]

I'm impressed with the FTS and the flexibility of it. Estimated (once off) Index build time was close to reality.

[BetterRed]

Assuming FTS doesn't index book 'data' folders, you could move the converted PDFs, LITs, PRCs etc into it, perhaps into a '4 Posterity' sub folder.

IIRC you can reindex individual books… but I can't remember how.

Found it: it's in the FTS search results, viz:



I don't use FTS, except on my Test library.

BR

[Quoth]

I don't have "converted" PDFs, just four sorts
1. Tech info, Service info (mostly not imported)
2. Manuals, instruction books etc that work OK on Sage
3. Scans of old books and magsines (mostly books, 100s of magasines not imported)
(The above three in no other format)

4. And finally, files for POD exported from LO Writer direct, for proofing layout/format only (never content) on Elipsa. Not many, but more than one size per title. These have a related epub for ebook publishing and azw3 and dual mobi purely for testing. The epub is from a docx, then deleted (original still exists in same place as odt). The azw3 & dual mobi from epub. The pdf not made by Calibre but a direct LO Writer export.

The only other "waste" in the indexing is where the epub is from a PD mobi or a converted epub or a bought azw3 converted to epub.
Hence 6337 vs 14249
There are also some titles with no formats.

Adding new books is no issue. Seems instant to index.

Free space Storage 2.9 TB
full-text-search-db = 10.6GB

The entire library as an export is about 23 GByte, so the FTS DB seems a bit large? Though Export is compressed.


Thanks for suggestions @BetterRed


I've no LITs, PDBs LRFs etc. The Palm Z22 was just a test

The FTS is very useful to me, not sure why I took so long to set it up.

Eventually a priority list of formats (like for transfer) would be an idea so that you could decide to only index one format per title.

(And rsync backed it up to my server already tonight. I never run the sync with Calibre or any editor /content creating program open).