Please implement faster index-search

[epubfan]

I have a library with 21.000 books and the index-search (title, author and so one) is very very slow (3 minutes and longer).
Any chance to implement a faster index-search-algorithm to calibre?

[theducks]

Originally Posted by epubfan

I have a library with 21.000 books and the index-search (title, author and so one) is very very slow (3 minutes and longer).
Any chance to implement a faster index-search-algorithm to calibre?

Your Anti-Virus is probably killing performance
OR
You are doing an unqualified search (Not specifying the search field) or adjusted the tweak to limit fields searched when not qualified.

[BetterRed]

Originally Posted by epubfan

I have a library with 21.000 books and the index-search (title, author and so one) is very very slow (3 minutes and longer).
Any chance to implement a faster index-search-algorithm to calibre?

I got faster results after removing comments from Search preferences - see attachment

BR

[epubfan]

OK, thanks.
Qualified search in 1 column is faster.
My default search before was "title, authors, tags, series, publisher".
But i am nevertheless a little confused that searches in 5 colums need so long time.

[theducks]

Originally Posted by epubfan

OK, thanks.
Qualified search in 1 column is faster.
My default search before was "title, authors, tags, series, publisher".
But i am nevertheless a little confused that searches in 5 colums need so long time.

IMHO Tags is the un-qualified search performance killer.
Each tag is a record below the skirts. How many different tags do you have?

[epubfan]

Originally Posted by theducks

IMHO Tags is the un-qualified search performance killer.
Each tag is a record below the skirts. How many different tags do you have?

active are title, author, date, size, words, language, format.

[BetterRed]

Originally Posted by epubfan

active are title, author, date, size, words, language, format.

I suspect format may slow things down because I think it refers to the extensions of files in the book folders

I thought there was a corresponding database column with a different name that was faster - but I've forgotten the name and I can't see it anywhere so maybe I dreamt it

BR

[PeterT]

Originally Posted by BetterRed

I suspect format may slow things down because I think it refers to the extensions of files in the book folders

I thought there was a corresponding database column with a different name that was faster - but I've forgotten the name and I can't see it anywhere so maybe I dreamt it

BR

Does this refresh your memory --> https://www.mobileread.com/forums/sho...d.php?t=188312

[BetterRed]

Originally Posted by PeterT

Does this refresh your memory --> https://www.mobileread.com/forums/sho...d.php?t=188312

- approximate_formats() is what I was thinking of, good to know I've better things than Calibre. perchance to dream upon

epubfan - suggest you do a search on approximate_formats to see what its all about - its a function rather than a column - you'll need to create a custom column - see attachment

BR

[PeterT]

Originally Posted by BetterRed

- approximate_formats() is what I was thinking of, good to know I've better things than Calibre of which to dream on

epubfan - suggest you do a search on approximate_formats to see what its all about - its a function rather than a column - I don't know for certain that functions can be used in searches but I'd be surprised if they can't.

BR

Just create a custom column based on other columns and then use that fiction call in the definition of the column. When I'm back at my PC will post how to do it.

[epubfan]

Originally Posted by PeterT

Just create a custom column based on other columns and then use that fiction call in the definition of the column. When I'm back at my PC will post how to do it.

Hm, then all informations in 1 db-column and searches will be faster or what is the trick?

[PeterT]

No idea... All I was trying to do was to aid with the possible format column issue. Try searching the calibre forums for other people with performance issues...

The column I refer can be added via Preferences | Interface | Add your own column

Lookup name:    approxformats
Column Heading: Rough List of Formats
Column Type:     Column built from other columns
Template:          {:'approximate_formats()'}

[BetterRed]

Originally Posted by epubfan

Hm, then all informations in 1 db-column and searches will be faster or what is the trick?

You wrote you search in title, author, date, size, words, language, format.

With the exception of format, these are database columns, the database is 'heavily indexed' in order to make searching and sorting fast.

However, 'formats' refers to the extensions of files in book folders, hence a search (or sort) must crawl through the library folder tree looking for files whose extensions match the search term, which is slow.

But, 'approximate_formats()' refers to a derived list of formats that you need to put in a custom database column which will be fast for the same reasons other database column based searches are.

See the link that Peter T posted for a discussion of why & how the function came to implemented. And see http://manual.calibre-ebook.com/temp...ximate-formats for definition and caveats

BR