Cleaning metadata (titles and authors)

[Starson17]

Originally Posted by chaley

1) Text mode -- case sensitive. Simple text search and replace.
2) Text mode -- case insensitive.
..... if you have any, speak now.

It looks like that will do everything I've wanted it to do.
Next up - the scripting language:

If (contents of field1 are 'XXXX"):
    Change field2 to "YYYY-XXX"

<wink>

Actually, I have a question about the quoted portion for regular searching (not the new advanced S&R). AFAICT, all Search Box searches are now case insensitive. There are times when I want to do "Text mode -- case sensitive" searches. Is there a way to do that now? If not, it would be useful to have a switch or checkbox for the regex search box to not ignore case or just a simple pull down option hidden away in the Advanced Search dialog for "Text mode -- case sensitive".
For your consideration.

[chaley]

Originally Posted by Starson17

It looks like that will do everything I've wanted it to do.
Next up - the scripting language:

If (contents of field1 are 'XXXX"):
    Change field2 to "YYYY-XXX"

<wink>

It is there, but you must use lisp in lambda notation.

Actually, I have a question about the quoted portion for regular searching (not the new advanced S&R). AFAICT, all Search Box searches are now case insensitive. There are times when I want to do "Text mode -- case sensitive" searches. Is there a way to do that now? If not, it would be useful to have a switch or checkbox for the regex search box to not ignore case or just a simple pull down option hidden away in the Advanced Search dialog for "Text mode -- case sensitive".
For your consideration.

It took me a while to find out how to disable case-insensitive matching in the new S/R function. I was going crazy entering \G in a box, and having it become \g because I had entered that once before.

It wouldn't be hard to provide a tweak that would make all the history comboboxes case sensitive. It would be much more if a bother to do them individually. Boxes where it really should work one way or the other (like S/R) could set the behavior explicitly.

I have finished the latest effort and submitted it to Kovid. The attached screen shot shows what it looks like in regexp mode. In character mode the 'destination' line of controls goes away. You will see that I did not do general templates, but instead provide for explicit source and destination fields. Handing multiplicity was simply too hard in the template case.

[Starson17]

Originally Posted by chaley

It wouldn't be hard to provide a tweak that would make all the history comboboxes case sensitive. It would be much more if a bother to do them individually. Boxes where it really should work one way or the other (like S/R) could set the behavior explicitly.

Up until now, I have had very few situations where I want case sensitive regular searches. I would not want a tweak to set it permanently, but OTOH, I wouldn't mind having to jump through a hoop or two to get that capability for the few times I need it for the search (however, see below). I don't have any need to save a case sensitive search, but just a box to turn it on/off would be fine, even if that box was hidden in the Advanced dialog).

I have finished the latest effort and submitted it to Kovid. The attached screen shot shows what it looks like in regexp mode. In character mode the 'destination' line of controls goes away. You will see that I did not do general templates, but instead provide for explicit source and destination fields. Handing multiplicity was simply too hard in the template case.

It really looks great to me. It's exactly what I had in mind, with a bit extra. I doubt I'll miss having general templates.

On the case sensitivity issue for general searching, the fact that S&R can do case sensitive searches will actually make it more likely that I'll want case sensitive regular searches.
Before running a regex in S&R, I've been running it in the regular search. It helps give me a better idea of what odd cases may crop up in a case sensitive S&R before actually committing to it. It also helps give me more control over the selected files that the S&R will be applied to. There's no point in selecting more books for the bulk metadata edit than you know need to get changed. Without case sensitivity in regular searches, you may have to select all books, even if you only want to change some that have a certain case problem in the fields.

(Last - congrats on getting the new custom column release into beta. I hope to look it over soon.)

[chaley]

Originally Posted by Starson17

Up until now, I have had very few situations where I want case sensitive regular searches. I would not want a tweak to set it permanently, but OTOH, I wouldn't mind having to jump through a hoop or two to get that capability for the few times I need it for the search (however, see below).

I was confused. I thought you were talking about how history is matched in the combo boxes, not how the search itself is run.

I have no problem making a tweak that makes regexp searches case sensitive. It cannot be on by default, but for those who want it, fine. I just glanced at the search code. It is doable.

If you set the tweak, you would be required to use the (?i) notation if you want case-insensitive matching.
[QUOTE
(Last - congrats on getting the new custom column release into beta. I hope to look it over soon.)[/QUOTE]I hope it works well for you.

[Starson17]

Originally Posted by chaley

I was confused. I thought you were talking about how history is matched in the combo boxes, not how the search itself is run.

I thought there might be some confusion

I have no problem making a tweak that makes regexp searches case sensitive. It cannot be on by default, but for those who want it, fine. I just glanced at the search code. It is doable. If you set the tweak, you would be required to use the (?i) notation if you want case-insensitive matching.

I'd live with that, but I'd prefer an option box or to have one of the pulldown search methods in the Advanced Search dialog screen for regular searches be to do a case sensitive search. I almost never want case sensitive, so constantly typing ?i (did you mean /i) would be enough of a pain that I'd have to leave it off and switch the tweak on for when I truly needed it. With an option box or pulldown search option, presumably saved searches would not work right unless the option box or pulldown search option was set the same as the search was created, but that doesn't bother me. I have never run into a need for case sensitive saved searches. I only need them when fixing metadata.

[kovidgoyal]

why not just use the case sensitive checkbox to control if the regex is case sensitive or not.

And make the history matching case sensitive for the search and replace boxes.

[chaley]

Originally Posted by kovidgoyal

why not just use the case sensitive checkbox to control if the regex is case sensitive or not.

And make the history matching case sensitive for the search and replace boxes.

Both of these are done.

Starson17 is talking about the standard search (the library search box), which is hardwired to be case insensitive. There is no easy way to pass an option along with a search, so a checkbox on advanced search won't work. I could use (?I) to invert the flag and also not do the source_text.lower() before applying the pattern, but that is a non-standard flag. It doesn't mean anything else, so it would work. It is unfortunate that python does not have the ability to invert case sensitivity in the middle of the regexp.

@starson17: python does not use the perl syntax '/i'. It uses (?iLmsux), where the letters specify the flag to be applied. The documentation is

(One or more letters from the set 'i', 'L', 'm', 's', 'u', 'x'.) The group matches the empty string; the letters set the corresponding flags: re.I (ignore case), re.L (locale dependent), re.M (multi-line), re.S (dot matches all), re.U (Unicode dependent), and re.X (verbose), for the entire regular expression. (The flags are described in Module Contents.) This is useful if you wish to include the flags as part of the regular expression, instead of passing a flag argument to the re.compile() function.

[kovidgoyal]

What's the use case for case sensitive searching in the main search box?

[Starson17]

Originally Posted by kovidgoyal

What's the use case for case sensitive searching in the main search box?

1) Fixing all those case errors in the title I got when doing automated metadata fetching before you could disable author/title overwrite. Most of the metadata fetched for title seems to be very bad. If overwrite title is on, the case will be destroyed.
2) It might also prove useful for those who've run into problems when dual booting a case sensitive vs. case insensitive OS. I've been struggling with the errors in case a lot.
3) Before doing a case sensitive S&R, it'd be useful to do a case sensitive main search. I don't want to select any more records than I have to before applying the new bulk metadata edit. (Although this case may boil down to the two above).

[chaley]

@kovid: note that he is talking about case-sensitive regexps from the search bar, not general text-match searches.

[kovidgoyal]

Ah, I see. Yeah you'd need extra UI to do this.

[Starson17]

Originally Posted by chaley

@starson17: python does not use the perl syntax '/i'. It uses (?iLmsux), where the letters specify the flag to be applied. The documentation is

(One or more letters from the set 'i', 'L', 'm', 's', 'u', 'x'.) The group matches the empty string; the letters set the corresponding flags: re.I (ignore case), re.L (locale dependent), re.M (multi-line), re.S (dot matches all), re.U (Unicode dependent), and re.X (verbose), for the entire regular expression. (The flags are described in Module Contents.) This is useful if you wish to include the flags as part of the regular expression, instead of passing a flag argument to the re.compile() function.

Thanks. A smattering of Perl has gotten me into trouble more than once. I hadn't actually run into the flags in Python outside the re.compile() function. And for anyone else, the manual is always helpful:
http://docs.python.org/library/re.html#re-syntax

[Manichean]

This is the official thread for discussion of the new S&R- feature, right? It's a great feature, I only have one question: In the series field, is it possible to access the series index? I like to have books that are part of a series named as "<series> <series_index> - <title>" and tried to accomplish that automatically, but I couldn't get the index to appear.

(Edit: I'm aware that I could do this in the send to device- preferences. But I'm using tags to specify on-device folders and, quite simply, I'm too lazy to change all that )

[chaley]

Originally Posted by Manichean

This is the official thread for discussion of the new S&R- feature, right? It's a great feature, I only have one question: In the series field, is it possible to access the series index? I like to have books that are part of a series named as "<series> <series_index> - <title>" and tried to accomplish that automatically, but I couldn't get the index to appear.

I am not sure what you are trying to do. Are you setting the title?

In any event, you get one source field and one destination field. Both must be text type. It sounds like you want two source fields, series and series_index? Or perhaps even three: title, series, and series index? This was what the 'input template' was for. As noted above, I didn't end up building that because of problems with multiple-value columns (tags, for example). The tags source field is not the same as a template containing {tags}. In the tags==source case, the expression is applied on a tag-by-tag basis. In the template case, the expression would be applied once across the entire set of tags. It it hard to explain the two different behaviors.

Do I have votes? Should the template be provided, even though {tags} is not the same as the 'tags' source field?

[DoctorOhh]

Originally Posted by chaley

In the tags==source case, the expression is applied on a tag-by-tag basis. In the template case, the expression would be applied once across the entire set of tags. It it hard to explain the two different behaviors.

Do I have votes? Should the template be provided, even though {tags} is not the same as the 'tags' source field?

I might be misreading this and I'm not as versed as y'all are but I think the source case is enough.