strange characters since one of the last updates

Allerwertester · 11-12-2021, 08:30 AM

Hi,

I don't know when it started, but calibre is mixing up some special characters used in german and other (European) languages in booktitles after importing them into calibre.

Here's a book title from Albert Camus (La mort heureuse) that seems to look ok, but the letter ü is shown in calibre as two seperate character, first the u followed by the two dots:

Der glückliche Tod

Here, in this text box the ü is shown as one character, but when I put the cursor behind it and start deleting it, I need two attempts. The first deletes the two dots, leaving the bare u that than can be deleted with a second attempt. You propably can try for yourself, but I don't know if the text box translates the text into a different caracter set after sending it.

The workaround now is, to delete both, the dots and the u and re-type a ü instead. It also affects ä and ö, etc.

As this is a new behaviour since one of the last updates, I guess it has something to do with one of the last changes made in calibre.

Thanks!

kovidgoyal · 11-12-2021, 10:16 AM

What OS? Copy pasting your example text shows it correctly for me. See screenshot

Allerwertester · 11-12-2021, 11:49 AM

Quote:

Originally Posted by kovidgoyal

What OS? Copy pasting your example text shows it correctly for me. See screenshot

Thanks for the quick reply and sorry for the missing and important details: Mac OS X. It happened in the last OS X 10.11 and also happens in the new OS X 10.12. As I wrote, it was ok a few calibre versions ago.

It looks like shown in the attached screenshot.

Not a big deal, as I can correct it, but it was ok a while ago.

kovidgoyal · 11-12-2021, 12:38 PM

Its showing up correctly for me on macOS Monterey too I'm afraid.

Allerwertester · 11-12-2021, 01:13 PM

Thanks!

It obviously has something to do with the font I chose. A while ago I switched my favorite font Avenir. When I change the font back to simple Arial, everything looks fine.

So it's either having the nice font with weird behavior or the not so nice look.

Thanks again!

Allerwertester · 11-12-2021, 01:21 PM

Same with Verdana, by the way: ü devided.

davidfor · 11-13-2021, 12:27 AM

What is the source of the title? I have seen problems like that with how some of the metadata source plugins download the title. It is how the site has it, but, something was needed to be added to the plugin to change how the results were encoded.

kovidgoyal · 11-13-2021, 01:51 AM

Its basically the unicode composed vs decomposed forms. It can be worked around by doing the normalization before passing the data to calibre, but the bug is really in whatever font/rendering system is rendering decomposed forms as separate glyphs.

Sarmat89 · 11-13-2021, 02:02 AM

Must be a bug in Qt. Combining characters should be processed as one unit when selecting/removing them.

Allerwertester · 11-13-2021, 04:34 AM

Quote:

Originally Posted by davidfor

What is the source of the title? I have seen problems like that with how some of the metadata source plugins download the title. It is how the site has it, but, something was needed to be added to the plugin to change how the results were encoded.

I bought the book and just added it to calibre, using the add-to-calibre-button.

Allerwertester · 11-13-2021, 04:37 AM

Quote:

Originally Posted by Sarmat89

Must be a bug in Qt. Combining characters should be processed as one unit when selecting/removing them.

Yes, as I wrote, deleting the default ü and re-typing it, fixes the problem. No big deal, but there is propably a way fixing it with the import procedure.

Allerwertester · 11-13-2021, 04:44 AM

Quote:

Originally Posted by kovidgoyal

Its basically the unicode composed vs decomposed forms. It can be worked around by doing the normalization before passing the data to calibre, but the bug is really in whatever font/rendering system is rendering decomposed forms as separate glyphs.

To me it's not really a bug, as there's no real harm done and it's easy to fix. I just wanted to tell about it. There's propably a way to add another step to the importing tool? I guess there's a reason, why calibre gets rid of those special characters in the epub's file names. The original file name had the ü.

Thanks!

Allerwertester · 11-13-2021, 04:47 AM

Quote:

Originally Posted by Sarmat89

Must be a bug in Qt. Combining characters should be processed as one unit when selecting/removing them.

I'm not a developer nor a coder, and I had to look up what Qt is. So I guess that's a hint for Kovid. Thanks!

11-12-2021, 08:30 AM	#1
Allerwertester Member Posts: 19 Karma: 10 Join Date: Sep 2021 Location: Germany Device: kobo forma	strange characters since one of the last updates Hi, I don't know when it started, but calibre is mixing up some special characters used in german and other (European) languages in booktitles after importing them into calibre. Here's a book title from Albert Camus (La mort heureuse) that seems to look ok, but the letter ü is shown in calibre as two seperate character, first the u followed by the two dots: Der glückliche Tod Here, in this text box the ü is shown as one character, but when I put the cursor behind it and start deleting it, I need two attempts. The first deletes the two dots, leaving the bare u that than can be deleted with a second attempt. You propably can try for yourself, but I don't know if the text box translates the text into a different caracter set after sending it. The workaround now is, to delete both, the dots and the u and re-type a ü instead. It also affects ä and ö, etc. As this is a new behaviour since one of the last updates, I guess it has something to do with one of the last changes made in calibre. Thanks!

11-12-2021, 10:16 AM	#2
kovidgoyal creator of calibre Posts: 45,339 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	What OS? Copy pasting your example text shows it correctly for me. See screenshot Attached Thumbnails

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
strange characters in epub book	irow	Library Management	2	11-14-2012 03:53 AM
I am getting strange characters when i load an ebook	irow	Library Management	7	09-18-2012 02:21 PM
Strange Characters â??s	mufc	Recipes	2	12-24-2010 06:25 PM
Export from Indesign = strange characters	luthar28	Workshop	6	08-16-2010 04:17 PM
strange characters	jomaweb	Calibre	0	01-27-2010 06:10 AM

11-12-2021, 12:38 PM	#4
kovidgoyal creator of calibre Posts: 45,339 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Its showing up correctly for me on macOS Monterey too I'm afraid.

11-12-2021, 01:13 PM	#5
Allerwertester Member Posts: 19 Karma: 10 Join Date: Sep 2021 Location: Germany Device: kobo forma	Thanks! It obviously has something to do with the font I chose. A while ago I switched my favorite font Avenir. When I change the font back to simple Arial, everything looks fine. So it's either having the nice font with weird behavior or the not so nice look. Thanks again!

11-12-2021, 01:21 PM	#6
Allerwertester Member Posts: 19 Karma: 10 Join Date: Sep 2021 Location: Germany Device: kobo forma	Same with Verdana, by the way: ü devided.

11-13-2021, 12:27 AM	#7
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303824 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	What is the source of the title? I have seen problems like that with how some of the metadata source plugins download the title. It is how the site has it, but, something was needed to be added to the plugin to change how the results were encoded.

11-13-2021, 01:51 AM	#8
kovidgoyal creator of calibre Posts: 45,339 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Its basically the unicode composed vs decomposed forms. It can be worked around by doing the normalization before passing the data to calibre, but the bug is really in whatever font/rendering system is rendering decomposed forms as separate glyphs.

11-13-2021, 02:02 AM	#9
Sarmat89 Fanatic Posts: 516 Karma: 2268308 Join Date: Nov 2015 Device: none	Must be a bug in Qt. Combining characters should be processed as one unit when selecting/removing them.

Advert

Advert