Regular expression search in authors

[kakkalla]

Hi there.

I am trying to find all the books in my calibre library that have the surname, then a comma, then the first name.

I have tried the following in the Advanced Search window. I select Regular Expression, and then in the drop down menu I select authors and then in the field I type:

[\w],\s[\w]

I have also tried [A-Za-Z], [A-Za-z]

and hit return. But this does not work.

Can someone please tell me how to achieve this?

Thank you.

[Manichean]

Originally Posted by kakkalla

[\w],\s[\w]

You cannot use escape sequences in sets, IIRC. Also, you forgot to use quantifiers. Using

\w+,\s\w+

should do what you want.

[chaley]

Two issues:

- literal backslashes must be escaped with a backslash.
- the comma is treated strangely, because the 'real' character in the field is a '|'. This is a bug.

The following works today.

authors:"~^\\w*\\, \\w"

I will fix the strange comma problem. After the fix is released, then the following will work

authors:"~^\\w*, \\w"

[Starson17]

Originally Posted by chaley

Two issues:

- literal backslashes must be escaped with a backslash.
- the comma is treated strangely, because the 'real' character in the field is a '|'. This is a bug.

The following works today.

authors:"~^\\w*\\, \\w"

If the 'real' character in the field is a '|', then why does that work?

I will fix the strange comma problem. After the fix is released, then the following will work

authors:"~^\\w*, \\w"

Can you use upper case \W as the negated word character class or [A-Z]? My recollection is that caps are all forced to lower, but I'd like to verify that.

[chaley]

Originally Posted by Starson17

If the 'real' character in the field is a '|', then why does that work?

Because the comma in the query is changed to a '|' when searching authors. The fix for the bug is to change the comma to r'\|' if regexp mode is being used.

Can you use upper case \W as the negated word character class or [A-Z]? My recollection is that caps are all forced to lower, but I'd like to verify that.

Query strings are not forced to lower in regexp mode. However, the regexp searches are caseless; ignore case is forced on.

[Starson17]

Originally Posted by chaley

Because the comma in the query is changed to a '|' when searching authors. The fix for the bug is to change the comma to r'\|' if regexp mode is being used.

Got it.

Query strings are not forced to lower in regexp mode. However, the regexp searches are caseless; ignore case is forced on.

So does that mean that \W as the negated word character class will work fine, but [A-Z] will match [a-z] characters in the ignore case search?
(Note: I edited your post to add a missing slash, so I could quote it.)

[chaley]

Originally Posted by Starson17

Got it.

So does that mean that \W as the negated word character class will work fine, but [A-Z] will match [a-z] characters in the ignore case search?

That is my expectation. Character classes that don't depend on case should work without surprise. Classes that do depend on case will match both cases, even if the class contains only one of them.

[kakkalla]

Thanks so much!

This:

authors:"~^\\w*\\, \\w"

works like charm. Thanks again.

However, I am confused. Why the two backslashes before the w? I have a reference guide that says (quote):

\d, \w and \s are shorthand character classes ... can be used inside and outside character classes.

So my thinking was that I need "any letter", followed by a comma, then a space, then any letter. Hence I should be able to use \w*,\s\w*. So despite the bug about the comma, this should work, shouldn't it.

[Manichean]

If I understood the matter correctly, you need to escape the backslashes for the search parser in order for them to stay in place as a single backslash for the regex parser.

[chaley]

Originally Posted by Manichean

If I understood the matter correctly, you need to escape the backslashes for the search parser in order for them to stay in place as a single backslash for the regex parser.

You have it right.

Backslashing things is always a problem because the backslash is often used to 'escape' other special characters by each piece of the processing chain. In calibre, there are two such pieces: the search language parser and the search engine. Each will process backslashes.

The string passed to search is first processed by the search language parser. Backslashes are used as escapes: for example a \" means that the quote is part of the query and not the end of a query segment. Following fairly universal rules, all escaping backslashes are removed after processing. As such, \w is processed, determined to mean 'w', and the backslash is removed. The result is passed to the search engine, where additional escape processing is done, for example for regular expressions.

Because of the dual processing, if you want to pass a real backslash to the search engine, you must escape it using a doubled backslash. Thus \\w instead of \w.