em dash conversion error

[ColMac]

I'm looking at a book in the calibre edit book panels

I have a lot of em dashes which have apparently converted wrongly., and on the right hand panel file preview, it is displaying as a small empty square. On my Kindle, it displays as a rectangle with two "0"s at the top, and 97 below that.

It is easy enough to replace each one I find, but in the HTML view, the em dash is simply displaying as a space. Unfortunately not all of these em dashes were at the end of a line and followed by a ".

In many cases, they are in the middle of a sentence. I may be able to find them by searching for a double space, but presumably it must be a different character.

Any suggestions on what I should search for please.

[retiredbiker]

That 0097 you see is the character you can't see - it is U+0097 END OF UNGUARDED AREA. If you put the cursor just after one of these in the editor and look down to the bottom right of the screen it should show you the character name.

So it is a little tricky, but you can actually select that invisible character with shift+Left Arrow, copy it, and paste it into the Find box. Then replace it with the em-dash.

There may be other punctuation done with some invisible arcane characters. I have found a very few old books completely punctuated with these bizarre things. I used to be able to "see" these characters on my PC as funny little pictures in some applications, but some update to my OS must have changed something, and they are now invisible everywhere, so it is more difficult.

[ColMac]

Many many thanks, for a perfect explanation, and so fast too. I had not spotted the character names at the bottom right, so that's a great tip for me too.

It turned out I had 55 of them, so it was a lot easier using the search & replace. (I had feared there might be more!)

[BetterRed]

@ColMac - Curious - what formats did you convert from and to?

The editor's Reports Tool can provide a list of the characters used in a book, I use it to determine if a book has any 'unusual' characters and track them down of necessary (it's a clickable list):



The list at the bottom, and individual cells (e.g the codepoint value), can be copied to the clipboard which can be handy at times.

BR

[retiredbiker]

I've only ever seen these in quite old books from an iffy Russian site. When I find one it is usually totally punctuated this way, and the character report looks like this, with the first five characters being the punctuation. I haven't seen one for a while; kept one epub as a curiosity; can't remember if they were in other formats as well.

[Karellen]

Originally Posted by BetterRed

The list at the bottom, and individual cells (e.g the codepoint value), can be copied to the clipboard which can be handy at times.

What is that string of characters at the bottom of the list - Withasg.eldrnobfwvp.....
Is that what can be copied? What do you then do with it?
Thanks

[BetterRed]

My 'handy at times' implies ad-hoc usage.

Like most things in calibre, the string at the bottom has a context menu.

Example: I pasted a string that had rupee ₹ and rial ﷼ symbols onto a sticky note and used it to update Transtools Insert Symbol tool.

BR

[Karellen]

Originally Posted by BetterRed

My 'handy at times' implies ad-hoc usage.

Like most things in calibre, the string at the bottom has a context menu.

Example: I pasted a string that had rupee ₹ and rial ﷼ symbols onto a sticky note and used it to update Transtools Insert Symbol tool.

BR

Sorry, I am not on the ball with this.

Yes I can copy this string using the context menu.

overid_csCImagJhnGTtlPHELNSB09581XRuUK.42Af,MywW6( )VxFpb3kj’qD©7O“Yz?”!-Q—;é‘:&–/…*Z

But what is it?
What is it used for?
It does not change as I select the different characters in the list above it, but it is on the same page so I guess it is somehow related to the character list, but I have no idea what it is.

[ColMac]

Originally Posted by BetterRed

@[B]what formats did you convert from and to?
BR

Unfortunately, I've no idea now, it's been in my library for months, maybe years. Maybe in hindsight, I didn't even actually convert it.

Originally Posted by BetterRed

@[B]

The editor's Reports Tool can provide a list of the characters used in a book, I use it to determine if a book has any 'unusual' characters and track them down of necessary (it's a clickable list)

BR

I've never used that, but it is very useful. Thanks

[Sarmat89]

Just replace "UTF-8" with "Latin-1" in the header of the problem file.

[DNSB]

Originally Posted by Sarmat89

Just replace "UTF-8" with "Latin-1" in the header of the problem file.

Sadly, an epub MUST use UTF-8 or UTF-16. Amazon is not happy with an epub sent to it for publishing via KDP that does not use UTF-8 though I've never tried sending one with UTF-16 to see if that also works.