Change case in a regular expression

[myki]

Greetings,

Is there a way to capitalize some text with regex in Calibre ?
I think I've read somewhere that it was possible in python, but not in Calibre.

My goal :
I am trying to improve my "save as" modele, trying to extract the name of the author_sort and capitalize it, and extract the firstname.
For example : from Christie, Agatha to CHRISTIE Agatha
For now, i built 2 personnal fields. It works but it would be better to work with regex...

Thanx for your help

[Sabardeyn]

Ok, I'm not the best person to be answering because I definitely cannot build the regex expression for you. But lets start the ball rolling with some general Q&A that might inspire others to offer some regex examples.

I think it is possible to perform the regex operation that you want, but it's going to require a complex expression depending on what kind of Lastname situation(s) you want to account for. What I mean is, do you have lastnames that include:

• hyphenated names (Smith-Johnson)
• multi-word names (Michelle Van de Camp)
• empty names (ie, Cher - has no last name)
• names writen in leetspeak (Thompson = +h0|\/|p50|\|)

Are there any other word boundary issues (finding the beginning/end of a name)? Because things like this will be the major issue in determining the correct expression to use.

Another issue is where in calibre do you plan to use this expression? In Add Books? Or in Edit Metadata? Somewhere else?

Does the ebook source follow a distinct, rigidly enforced pattern? For instance a filename like: Title - Firstname Lastname.Extension. Consistent source material helps during input.



PS: Take a look at the topic Tyranosaurus Regex, including my post (msg #17) where the efforts of others came together in a wonderful regex expression which was particularly efficient. I didn't create it, I just stuck it all together (and I got lucky!).

There also used to be a Regex example topic, but I'm not seeing it ATM.

[eschwartz]

Use this, in general program mode. Or tweak into another mode, but general program mode is awesome for complex stuff, like combining multiple fields in one using complex regexes and function calls.

It will only parse the first author.

program:

	FN=re(
		field('author_sort'),
		'([^,]+),.+',
		'\1'
	);

	LN=re(
		field('author_sort'),
		'[^,]+,(.+)',
		'\1'
	);

	fixed_author_sort=strcat(
		uppercase(FN),
		LN
	)

[chaley]

This is an interesting case. In effect, it wants a function that allows a template to be applied to each group matched in the search regular expression, in effect providing similar functionality as python's match groups. This function, having a variable number of arguments, would look something like:

re_groups(field, search_expr, template_for_group_1, t_for_g_2, ...)

The group would be named '$' when the template for that group is called. The result is the field with each match of 'search_expr' replaced by the result of evaluating the template for each group and concatenating the groups.

For the situation being discussed in this thread and using eschwartz's example, one would use something like

re_groups(field('author_sort'), '([^,]+), (.+)', 'program: uppercase($)', 'program: $')

or

re_groups(field('author_sort'), '([^,]+), (.+)', "[[$:uppercase()]]", '[[$]]')

I will look at whether such a function is buildable.

[eschwartz]

Sounds awesome, chaley!

I gave up pretty quickly looking at that and took the easy way.

[chaley]

I have it working and will submit it for the next release.

Examples using your suggested approach:

program: re_group(field('authors'), '(\S*), (\S*)', '[[$:uppercase()]] ', "[[$]]")

or

{authors:'re_group(field('authors'), '(\S*), (\S*)', '[[$:uppercase()]] ', "[[$]]")'}

Now I need to add another function like list_re that uses re_group instead of re. Something like

list_re_group(src_list, separator, search_re, group_1, group_2, ...)

that would apply the re_group to each item in the list.

[myki]

Thanx for your help guys... But you lost me lol
I don't know anything about the program mode.
I looked into the documentation and it frightened me !

I don't want to make you lose your time, but if you're still motivated to help me, i would need more explantions, beginner oriented please

Thank you !

HS : Calibre can do so much things, it's really impressive.

EDIT : Argh ! because of my english, i have confused "capitalize" and "change in uppercase", from Christie, Agatha to CHRISTIE Agatha.
Sorry for that !

[eschwartz]

Currently you can use my program: block to "fix" the author_sort for your use. I would recommend using it inside {:'program-goes-here'} e.g.

{:'program:FN=re(field("author_sort"),"([^,]+),.+","\1");LN=re(field("author_sort"),"[^,]+,(.+)",\1");fixed_author_sort=strcat(uppercase(FN),LN)'}

can be used as a template to produce author_sort_fixed.

Either code block works, but:

• This new one is stuck into a template, allowing you to follow it with more standard templates.
• The previous one was meant for integrating into program mode, where any further sections have to be called with the field function and strcat'ed onto the last function call. If you don't understand this, you don't have to -- just use the new one.

Using general program mode is only required for people who want to spend time making it look nice. And are probably a little obsessed about it. As you can see, the code I just gave is a long incomprehensible blob without the neatly-laid-out whitespace and stuff. But it works just as well once written.

Also, you can use chaley's new function if you are willing to wait until Friday (and the next calibre release).

[eschwartz]

Originally Posted by myki

EDIT : Argh ! because of my english, i have confused "capitalize" and "change in uppercase", from Christie, Agatha to CHRISTIE Agatha.
Sorry for that !

Your intent was clear, though. for giving an example of what you want.

Also, technically you are capitalizing all letters.

[mrmikel]

Capitalizing only the first letter of each word is often referred to as title case.

If it makes you feel any better, I don't know anything about Python (the language the program is written in) either.

[myki]

Thank you very much, it is very instructive !

I hesitate to change the way "author_sort" works, so finally i found a solution :
I built 2 personnalised columns, from another :

#author_sort_lastname with {:'uppercase('{author_sort:list_item(0,\,)}')'}
#author_sort_firstname with {author_sort:list_item(1,\,)}

And my save as modele :

{#author_sort_lastname} {#author_sort_firstname:re([.],)}/{series:'test($,'{series}/[{series}-{series_index:0>2s}] ','{#author_sort_lastname} {#author_sort_firstname} - ')'}{title}

I kept first and last name separated in case i wish in the future, the firstname before

In the adjustments, i changed :

save_template_title_series_sorting = 'strictly_alphabetic'

At last, i tried to replace {#author_sort_lastname} and {#author_sort_firstname} by their value, but i get an error.

It was very interesting, thanx again

[chaley]

Originally Posted by myki

Thank you very much, it is very instructive !
...

#author_sort_lastname with {:'uppercase('{author_sort:list_item(0,\,)}')'}

And my save as modele :

{#author_sort_lastname} {#author_sort_firstname:re([.],)}/{series:'test($,'{series}/[{series}-{series_index:0>2s}] ','{#author_sort_lastname} {#author_sort_firstname} - ')'}{title}

I note that you are using subtemplates (nested templates). You should not do that. There are several ways that evaluating templates can break by using subtemplates.

Your first template is better written as something like

{author_sort:'uppercase(list_item($, 0,\,))'}

and your second one should really be written in general program mode because of its complexity, something like:

program:
	as = field('author_sort');
# list item removes the comma. Note that this doesn't work if there 
# are multiple authors or if the author name doesn't contain a comma
	asfn = list_item(as, 1, ',');
	asln = uppercase(list_item(as, 0, ','));

# Use the template function as a convenience to avoid calling format_number
	has_series = template('{series}/[{series}-{series_index:0>2s}]');
	no_series = strcat(asln, ' ', asfn, '  -');
	series_val = test(field('series'), has_series, no_series);

	strcat(asln, ' ', asfn,  '/', series_val, ' ', field('title'))

Note that this template does not make use of the #author_sort_*_name columns, so perhaps they can go away.

[chaley]

Originally Posted by eschwartz

Using general program mode is only required for people who want to spend time making it look nice. And are probably a little obsessed about it.

Not quite true. GPM is close to required when using subtemplates. One *can* use a form of subtemplates in TPM, but it is very tricky and error prone.

[chaley]

Originally Posted by chaley

Now I need to add another function like list_re that uses re_group instead of re. Something like

list_re_group(src_list, separator, search_re, group_1, group_2, ...)

that would apply the re_group to each item in the list.

I submitted this function as well. The prototype is different from what I suggested earlier:

list_re_group(src_list, separator, include_re, search_re, group_1, group_2, ...)

The function first builds a result list by applying 'include_re' against each list element, then applies re_group(search_re, g1, ...) to each resulting element.

This thread provides an example of this function's use, to uppercase the last name of each author for a book:

program: list_re_group(field('authors'), ' & ', '.', '([^,]*), (.*)', '{$:uppercase()}, ', '{$}')

or

{authors:'list_re_group($, ' & ', '.', '([^,]*), (.*)', '[[$:uppercase()]], ', '[[$]]')'}

[myki]

As i said i didn't knew anything about GPM.
For me i feel more comfortable with program mode thanks to the use of variables.
And the writing is more vertical, when the normal mode is more horizontal.

I'm curious :
Is it possible to use variables in normal mode, and how ??
Is it possible to increase the size of the police inside the modele editor ?

Thanx for your help...

@chaley :
If i had your function, it would have been so easier for me to harmonize my authors the first time i built my database