Calibre's search is pretty lacking and has issues

[BetterRed]

@chaley - what is the source of the data in the tree?

BR

[chaley]

Originally Posted by BetterRed

@chaley - what is the source of the data in the tree?

BR

The query expression, possibly composed with the restriction expression(s). It uses a recursive descent parser. Saved searches are expanded when encountered.

The grammar is very simple.

prog ::= or_expression
or_expression ::= and_expression [ 'or' or_expression ]
and_expression ::= not_expression [ [ 'and' ] and_expression ]
not_expression ::= [ 'not' ] location_expression
location_expression ::= base_token | ( '(' or_expression ')' )
base_token ::= a sequence of letters and colons, perhaps quoted

The base_token node is actually more complex than the grammar would indicate. The parser separates out the column name and deals with quotes.

However, you might be asking about the source of the data for the books. That is, of course, the database, but by the time search sees the data, it has been flattened into a single table representation, one row per book, every field in its own column even if that field contains multiple values. Only the very bottom layers of calibre know anything about the database and its underlying table structure.

[MelBr]

Good discussion guys! I see there's a lot of things to think about!

Here's one other example of current Calibre's search problems. I encountered this issue few days ago and I had no idea what was wrong until I spent few minutes investigating.

Say your title (or whatever filed you wish… description filed with rich text is usually the culprit here) uses a proper curly quote, ’ , and not ' as an apostrophe…. as in: "Calibre’s Search". And say you want to search for it and you type "calibre's search" (regular apostrophe). If you followed discussion so far, you're realize that Calibre won't show you the book. But what do you think will happen when you copy paste the exact text string into the search field: "Calibre’s Search"? Will Calibre find it? What do you think? No cheating

This why it's important to strip all these characters out. Someone who's not aware of these issues will never figure out why they can't find anything. When importing books in to Calibre, it shouldn’t feel like I'm putting things into a black hole .

[BetterRed]

@MelBR - you can read about the folks working on the apostrophe problem here

If you strip out the apostrophes ( ’ ) and quotes ( ' ) when they're used to indicate possession then you should probably remove any following ess ( s ). That leaves one with the problem of what to do about them when they’re used within a contraction. Then there's chaley's point of having to deal with 100's [but you wouldn't remove that ( s )] of languages on several operating systems in different versions and guises.

If you're the NSA there's no such thing as a black hole

@chaley - as you surmised I was thinking of the metadata database. If none of its columns contains the value I'm looking for then no amount of 'smarts' within calibre's search facility will find it. Or if a column erroneously contains what I'm looking for, then search will unfortunately find it. In both cases I end up with the wrong result.

But that said, I don't have any complaints about calibre's search. My expectations of it are limited, I mainly use it for administrative searches, eg 'find' books that don't have an RTF or a DOCX or a PDF. Otherwise I use one of several external full text search tools, sometimes from there I'll create a CSV to read into Import List to create a Reading List.

BR

[speakingtohe]

I often use the tag browser find to search for the author. Very fast usually even with partial names.

[MelBr]

Originally Posted by BetterRed

If you strip out the apostrophes ( ’ ) and quotes ( ' ) when they're used to indicate possession then you should probably remove any following ess ( s ). That leaves one with the problem of what to do about them when they’re used within a contraction. Then there's chaley's point of having to deal with 100's [but you wouldn't remove that ( s )] of languages on several operating systems in different versions and guises.

If you're the NSA there's no such thing as a black hole

And yet if you leave it the way it is now, you can't even search for the exact string inside of Calibre. Even if you copy paste a string containing some chars, Calibre will return nothing.

Originally Posted by speakingtohe

I often use the tag browser find to search for the author. Very fast usually even with partial names.

That might be fine for fiction or small libraries but it's unrealistic for non-fiction and large libraries. Not to mention that it's not user-friendly.

[BetterRed]

Originally Posted by MelBr

And yet if you leave it the way it is now, you can't even search for the exact string inside of Calibre. Even if you copy paste a string containing some chars, Calibre will return nothing.

That's more or less why I limit my use of calibre to do administrative searches, and use external tools to do informational searches.

Originally Posted by MelBr

That might be fine for fiction or small libraries but it's unrealistic for non-fiction and large libraries. Not to mention that it's not user-friendly.

I suggest calibre's primary target audience is people who read books from cover to cover - most of which is fiction and biographies. IMO if you use calibre on an ebook reference library you have to recognise the limitations that brings with it, one of which is limited informational searching.

And its not entirely useless in that regard. If I want to find a article that I think Stiglitz wrote on the IMF blog, then I type 'stiglitz' in the calibre search bar, sort on Publisher and look at the Titles**. But if I want to find what I have by Stiglitz where he's had something to say about CoCo Bonds then I will use Windows Search or Recoll.

BR

** if you're wondering why I wouldn't just go the IMF blog, well just because something was there yesterday doesn't mean it'll be there next week.

[Sefiriot]

Originally Posted by BetterRed

That's more or less why I limit my use of calibre to do administrative searches, and use external tools to do informational searches.


I suggest calibre's primary target audience is people who read books from cover to cover - most of which is fiction and biographies. IMO if you use calibre on an ebook reference library you have to recognise the limitations that brings with it, one of which is limited informational searching.

And its not entirely useless in that regard. If I want to find a article that I think Stiglitz wrote on the IMF blog, then I type 'stiglitz' in the calibre search bar, sort on Publisher and look at the Titles**. But if I want to find what I have by Stiglitz where he's had something to say about CoCo Bonds then I will use Windows Search or Recoll.

BR

** if you're wondering why I wouldn't just go the IMF blog, well just because something was there yesterday doesn't mean it'll be there next week.

And even the CoCo Bonds could probably be searchable, assuming you're extremely diligent about your metadata entry and policing your tags with each new addition. A quick skim or search on the text for keywords you deem important and you could use those to tag and search through Calibre.

[BetterRed]

Originally Posted by Sefiriot

assuming you're extremely diligent about your metadata entry and policing your tags with each new addition. A quick skim or search on the text for keywords

You assume that what I deem to be unworthy of tagging today will remain so forever. When you're dealing with current events in finance and economics it ain't so. I never imagined in the mid nineties that credit default swaps which had just been devised (divined, confected) at JP Morgan, would become a hot topic when the Greek sovereign debt issue blew up 15 years later. If I had that level of prescience I'd be running JP Morgan

Full Text indexers although mechanistic don't make value judgements, they skim read fast and they tag 'everything'. Even the ones on desktop systems are 'smarter' and 'faster' than I will ever be. But they do the 'easy bit' the 'hard bit' is knowing how to search and more importantly how to 'join the dots'.

br