visibility of soft-hyphens

[DrChiper]

I was editing and informed about a incorrectly spelled word (curly underline). However, it looked ok. Changing it via the suggested word (which seems to be exactly the same word) did removed the curly underline, so something was replaced somehow. I inspected the original word letter by letter and noticed that there was an invisible soft-hyphen placed between two letters.

It would be nice to have that soft-hyphen made visible, but apparently there is no editor setting to do so. Changing colorschemas does not help.
So, is it even possible to show them? In the current situation you must explicitly search for them to find them. A visible indication would be nice.

[BetterRed]

I use Tools->Reports->Characters to determine if any 'interesting' characters exist. I can then step though through the relevant code char-by-char watching the bottom left corner - I prefer not to delete/replace them en-masse.

Sigil has a plugin to help deal with Shady Characters. Perhaps DiapDealer could package it as a calibre editor plugin, as he has with his Toolbag plugin.

I too couldn't find a way of highlighting 'invisible characters' via colour scheme. Would be nice to have a toggle to show 'invisible characters' as a user configured character/colour, eg ✘.

BR

[phossler]

Since I discovered the Hyphenate PI, I changed my workflow

I've starting running a stored F&R to remove any soft hyphens until I'm done editing, etc.

Then I use the PI to put them back, polish, etc. and convert the epub to azw3 format.

[kovidgoyal]

There is no way to show invisible characters that take no space in the underlying Qt widget. Your best bet is to simply run a search replace to replace all instances of such characters with something visible, like one of the unicode block characters.

[phossler]

Originally Posted by kovidgoyal

There is no way to show invisible characters that take no space in the underlying Qt widget. Your best bet is to simply run a search replace to replace all instances of such characters with something visible, like one of the unicode block characters.

Since the soft hyphens cause a word to be listed as mis-spelt -- generating a lot more review effort -- replacing with another character would most likely also

FWIW, that's why I decided to just delete the soft hyphens until the very end

[DrChiper]

Or perhaps let the dictionary ignore just any soft-hyphens, which might also do the trick. Normally, you would not notice the soft-hyphens, but the dictionary check revealed them as being (part of) misspelled words, which is not actually true. Soft-hyphens are just "hints" for any viewer to "break" words according some spelling rules.

[kovidgoyal]

There are over a dozen invisible characters not just soft-hyphens. zero-width spaces, various special chars in various scripts, etc. And just special casing all of them int he spell checking code is not the answer -- they will still cause problems in many other places, such as search and replace.

[quiris]

RFC 2070 - Internationalization of the Hypertext Markup Language on this matter: http://www.faqs.org/rfcs/rfc2070.html

NOTE - the soft hyphen character (U+00AD) needs special attention from user-agent implementers. It is present in many character sets (including the whole ISO 8859 series and, of course, ISO 10646), and can always be included by means of the reference ­. Its semantics are different from the plain hyphen: it indicates a point in a word where a line break is allowed. If the line is indeed broken there, a hyphen must be displayed at the end of the first line. If not, the character is not displayed at all. In operations like searching and sorting, it must always be ignored.

And also HTML 4.01: http://www.w3.org/TR/html4/struct/text.html#h-9.3.3

In HTML, there are two types of hyphens: the plain hyphen and the soft hyphen. The plain hyphen should be interpreted by a user agent as just another character. The soft hyphen tells the user agent where a line break can occur.

Those browsers that interpret soft hyphens must observe the following semantics: If a line is broken at a soft hyphen, a hyphen character must be displayed at the end of the first line. If a line is not broken at a soft hyphen, the user agent must not display a hyphen character. For operations such as searching and sorting, the soft hyphen should always be ignored.

[Phssthpok]

I also use a regex function that turns chars into entities, which is probably the best way to make dodgy chars visible in a sensible way; see https://www.mobileread.com/forums/sho...d.php?t=270821

[marvin_2]

The hyphenate-plugin can remove and add soft-hyphens. It uses OpenOffice dictionaries, I use it regularly and have yet to find the first error.