strange invisible? character - how to find it?

[stumped]

so I recently posted about an issue with calibre content server not giving me recent tweak of a specific book.
what led to this is that in this book, after any scene break, I am seeing a strange artifact in Moon reader. I see the letters OBJ surrounded by a square of dashes forming a border.
I see nothing at all in calibre or sigil editors to account for this
but when I view the book in adobe digital editions, I see an X inside a square as the strange character.

messing with scene breaks was a try at removing stuff that might cause it but no joy

so I have an "invisible" character at those points in the epub, now how can I edit it out ?
invisible in sigil, invisible in calibre editor, visible in ADE, visible in notepad ++, visible in Moon reader
NB if i paste it into MS word, I see the same artifact as in Moon reader i.e. OBJ inside of a border

in calibre editor I see e.g.
<p class="x03-co-body-text">The ...

with nothing untoward about that class

but when I paste a snippet into notepad++ I see the "invisible" character as a thinck horizontal dash with dotted lines above and below. the strange character is after the > and before the T in the above example
so there is definitely something there, but how can I edit it away if I cant see it in the editor.
i could probably write some regex to work on the 1st character after that specific x03 class as that's always where it happens, But I am curious as to what it is and why it is there ?



[DiapDealer]

I was under the impression that calibre's editor had some sort of visual indication that invisible characters were present. A yellow color or something?

It's probably a soft hyphen or a zero-width-joiner, or the like.

[stumped]

Originally Posted by DiapDealer

I was under the impression that calibre's editor had some sort of visual indication that invisible characters were present. A yellow color or something?

It's probably a soft hyphen or a zero-width-joiner, or the like.

Maybe, i also thought it would be easy in notepad++ to get it to tell me the actual ANSI or whatever code as a 0 - 255 value . but when I select ANSI in place of UTF I see 3 or more squiggles
note also, this is the first characxter of the 1st sentence after each scene break - not somewhere you'd put a hyphen of any kind ?.
the actual structure, before I ripped it all out was a <HR class = "transition" >
adding a thin horizontal line
the a scene break ornament picture thingy -in <img tags
then onto the next scene.

i can live with it as it, but i obstinately still want to identify it , then zap it via regex, if possible.

IN SIGIL - I THINK
find
<p class="x03-co-body-text">[^A-Z]
replace
<p class="x03-co-body-text">
will zap it
but then I will never know what it was

[kovidgoyal]

simply move your cursor around it, you will see it in the bottom right corner of the editor screen as you do

[theducks]

Originally Posted by DiapDealer

I was under the impression that calibre's editor had some sort of visual indication that invisible characters were present. A yellow color or something?

The editor shows that (the char name) in the lower Right corner

[stumped]

Thanks. Library pc is off for the night so I will have another look in the editor in the morning.
Meanwhile, idly googling the issue from tablet,, I wonder if it is some format control character used in word and somehow left in.
Something that might be desirable for starting a new paragraph after a scene break.

I found a website what said insert your invisible character here... And got it out of moon reader and onto the clipboard but no visible result
Another link suggested it might be a 144 control code, as they can appear as an X in a box.

Staying with the calibre editor, as a learning exercise. Let's say I do spot it in the right hand corner. Can I then get it into regex find command , in the editor , to do a find all, replace with nothing.... Regex?

[DiapDealer]

Originally Posted by theducks

The editor shows that (the char name) in the lower Right corner

I knew it showed the char name next to the cursor somewhere, but I could have sworn there was some sort of yellow(?) background that was a visual indicator of where an invisible character was lurking. But maybe that was just talked about RE someone's wishlist of features, or something. Who knows?

[stumped]

In sigil the only clue was no regex matches when I ran find on the opening style followed by a capital letters which is what was visible. So something between the close bracket of the style and the 1st visible letter was affecting regex find.

[DiapDealer]

Yes. Without the actual character present to copy, you'd need to use some form of hex character code with regex to find it. In Sigil searching for \x{200D} would find a zero-width joiner character (hex code 0x200D).

x\{00AD} would find soft-hyphens
x\{202F} - narrow non-breaking space
x\{FEFF} - zero-width non-breaking space

Can't remember if calibre's regex engine uses the same notation or not. Could be something like u\{00AD} or u\00AD

[BetterRed]

Originally Posted by DiapDealer

I knew it showed the char name next to the cursor somewhere, but I could have sworn there was some sort of yellow(?) background that was a visual indicator of where an invisible character was lurking. But maybe that was just talked about RE someone's wishlist of features, or something. Who knows?

Me - you're memory is spot-on

Characters such as nbsp, em-dash etc are highlighted with the "Special Character" colour - which is 'yellowish' by default.

The Tools->Reports->Characters tool can be useful in situations like this.

BR

[DiapDealer]

That was it! Thanks.

It makes sense that the feature doesn't extend to special invisible characters now that I think about it. Pretty hard to highlight something you can't see.

[BetterRed]

Originally Posted by DiapDealer

Pretty hard to highlight something you can't see.

Especially when the something you can't see gets trumped by the something you can see ==>> https://www.mobileread.com/forums/sh...d.php?t=313380

BR

[stumped]

ok - after moving the cursor back and forth I got the invisible character named in the editor.
I had to put cursor on the 1st visible letter , then back arrow to get it to show at bottom right
the description then says Object Replacement Character
so I am none the wiser as to how/why it got to be there, but I reckon I can zap it with my previous planned regex, using not A-Z to track it down, I probably need not A-Z or " to cover all start of sentence possibilities

here is a wiki entry: for object replacement character
https://en.wiktionary.org/wiki/%EF%BF%BC

(computing) The object replacement character, sometimes used to represent an embedded object in a document when it is converted to plain text.

[DiapDealer]

Find: \uFFFC
Replace with: ""

[stumped]

thanks, but I have done it now, using a not A-Z workaround. looks OK