Plugboard: Removing Unicode characters that Kobo can't render

[ownedbycats]

I have this plugboard for "series" in KOBOTOUCHEXTENDED:

{series:'re_group($,"([^.]+)\.","[[$]]: ")'}

I forgot exactly why I'm using the re_group.

Anyways, I have a fanfic that has some fancy unicode characters for some effect:



This is how it appears on the Kobo:



I'm guessing there's no real way for the Kobo to deal with this, so is there anything I can do on Calibre side without changing the series? Is there a way to strip out the Unicode without leaving behind the dash and I?

[chaley]

Could you give us the actual source string, not an image of it? And could you give us what you want as the destination string?

A brute force solution would be to use a plugboard that looks for offending series names and returns the desired series name. This isn't too painful if you don't have many of them.

EDIT: I wonder what the 'transliterate' formatter function would do with your string ...

[ownedbycats]

Assuming it posts correctly on the forum, the characters are ᕼ ᗩ ᒪ ᖴ - ᒪ I ᖴ ᗴ.

[chaley]

Well, 'transliterate' doesn't do anything useful. For the string ''ᕼ ᗩ ᒪ ᖴ - ᒪ I ᖴ ᗴ'' it returns 'h po ma we - ma I we ga'.

[chaley]

A example plugboard template that returns what I think you want follows. It returns a special string if the series matches some value, otherwise it returns the series. If the series is empty it returns the empty string.

program:
	s = $series;
#	s = 'ᕼ ᗩ ᒪ ᖴ - ᒪ I ᖴ ᗴ';
	first_non_empty(
		if s == 'ᕼ ᗩ ᒪ ᖴ - ᒪ I ᖴ ᗴ' then 'Half-life' fi,
# 		add more lines here if needed
		s
	)

[capink]

You can also use user defined template functions to do one of two things:

1. Remove non unicode characters as you requested:
   

   def evaluate(self, formatter, kwargs, mi, locals, val):
   	string_encode = val.encode("ascii", "ignore")
   	string_decode = string_encode.decode()
   	return string_decode

   you can use the new function like this:
   
   

   program:
   	s = 'ᕼ ᗩ ᒪ ᖴ - ᒪ I ᖴ ᗴ';
   	remove_unicode_characters(s)

2. Make lookup replacement for unicode characters:
   

   def evaluate(self, formatter, kwargs, mi, locals, val):
   	lookup = {
   	    'ᕼ': 'H',
   	    'ᗩ': 'A',
   	    'ᒪ': 'L',
   	    'ᖴ': 'F',
   	    'ᗴ': 'E'
   	}
   	new_string = ''
   	for ch in val:
   	    new_string += lookup.get(ch, ch)
   	return new_string

   You need to expand the lookup dictionary (highlighted in blue) to include any additional unicode characters you want.
   
   You can use this new function as follows
   
   

   program:
   	s = 'ᕼ ᗩ ᒪ ᖴ - ᒪ I ᖴ ᗴ';
   	unicode_lookup(s)

Notes

• To add the new functions described above: preferences > template functions > template functions tab > copy-paste in the program code box and enter the name of the function (highlighted above in green) in the function box.
• For both functions the argument count should be set to 1.

[ownedbycats]

Thanks