Bug with zero-width space Unicode

[Peter Ahlstrom]

Well, I wanted to officially submit an issue, but the Sigil website says I should just fix the bug myself and submit a patch. As I'm no programmer, I guess this is the next best thing!

I've got a book with a few paragraphs that are meant to have no spaces in them. I told the ebook formatting people to use zero-width spaces between where the word boundaries would normally be, so the lines still wrap. Here is what the html looks like in a text editor:

< p>Ah& #8203;​but& #8203;​they& #8203;were& #8203;left& #8203;behind& #8203;It& #8203;is& #8203;obvious& #8203;from& #8203;then& #8203;ature& #8203;of& #8203;the& #8203;bond& #8203;But& #8203;where& #8203;where& #8203;where& #8203;where& #8203;Setoff& #8203;Obvious& #8203;Realization& #8203;like& #8203;a& #8203;pricity& #8203;They& #8203;are& #8203;with& #8203;the& #8203;Shin& #8203;We& #8203;must& #8203;find& #8203;one& #8203;Can& #8203;we& #8203;make & #8203;to& #8203;use& #8203;a& #8203;Truthless& #8203;Can& #8203;we& #8203;craft& #8203;a& #8203;weapon< /p>

(I put a space in each html entity and tag so it will display here, but there's no space in the html itself.)

This looks perfect in all ebook readers, except for in amzn-mobi, for which we get around the issue with a media query.

The problem is how it displays in Sigil. Here is how that paragraph displays in Sigil, in the html view (not wysiwyg):

< p>Ah​but​they​were​left​behind​It​is​obvious​from​ then​ature​​ض​the​bond​But​where​where​where​wher e​Setoff​/¢∂ص≥​Realization​like​a​pricity​4®•π​are​with​the ​Shin​We​must​find​one​Can​we​make​to​use​a​Truthl ess#°Æ​we​craft​a​weapon< /p>

So as you can see, some of the individual words get changed to garbage characters. (Also, the forum software here is adding some spaces.)

However, this is only the way it displays. The underlying text looks normal—Sigil is converting all of the zero-width html entities to actual zero-width Unicode characters. And the intervening characters that look like garbage above are not actually garbage—if you save the file and look at it in a text editor, those characters look fine. And if I look at it in a hex editor, each zero-width space is E2808B exactly as expected for zero-width space Unicode.

But this is not very useful in Sigil—it looks buggy as I mentioned above, and it runs all the text together as if the zero-width spaces aren't there. (That's what it's supposed to look like, facing the end user—not in the html code, which should give an indication that the zero-width spaces are there so an editor can do something with them.)

[Toxaris]

This is most probably a bug in QT, the underlying framework. If so, there is not much you can do. How does it look in the Calibre viewer?

[Peter Ahlstrom]

Even if there is a bug in QT, I think a better default behavior would be for & #8203; to be left as-is and not converted to straight Unicode, the same way & #160; is left as-is.

[DiapDealer]

I don't believe there's going to be any modifications to Sigil anytime soon. It's at a very "if -it-doesn't-work-for-you-as-is-(and you can't change it yourself)-you're-probably-going-to-want-to-use-something-else" stage in it's development.

[Hitch]

Originally Posted by Peter Ahlstrom

Well, I wanted to officially submit an issue, but the Sigil website says I should just fix the bug myself and submit a patch. As I'm no programmer, I guess this is the next best thing!

I've got a book with a few paragraphs that are meant to have no spaces in them. I told the ebook formatting people to use zero-width spaces between where the word boundaries would normally be, so the lines still wrap. <snip>

But this is not very useful in Sigil—it looks buggy as I mentioned above, and it runs all the text together as if the zero-width spaces aren't there. (That's what it's supposed to look like, facing the end user—not in the html code, which should give an indication that the zero-width spaces are there so an editor can do something with them.)

And people wonder why Commercial converters have a high asylum commitment rate, or simply self-medicate with Jack Daniels.

Hitch

[mrmikel]

Just before they gave up, the developers of Sigil seesawed back and forth on how to do this and I don't think that anyone was happy with any iteration of it. If it works for one group, it doesn't for another, particularly if they are dumping web page junk into it.

[Peter Ahlstrom]

"Just before they gave up"? Oh. I wasn't aware it's essentially abandoned. Well, it does work quite well for what it does. I guess if I need zero-width spaces in the future, I'll use BBEdit for those bits.

[eschwartz]

Making epub happen: Sigil's Spiritual Successor

At this point Sigil is no longer being actively developed. Moving development to Github has netted a few contributions but they were one offs and fairly minor. With Sigil development being stalled, Kovid (of calibre) starting making the tweak epub functionality in calibre into a full editor.

calibre's editor at this point is stable and has many of the features, though not all (yet), that are present in Sigil. Like Sigil, calibre's editor is open source and unlike Sigil is being actively developed. I've known Kovid for quite some time (though calibre) and I'm confident that the calibre editor is the way forward.

For people using Sigil, keep using it as long as it works for you. If you find it's not meeting your needs or if you want to see what else is out there I recommend checking out cailbre's editor. While it doesn't use any of Sigil's code I consider it Sigil's spiritual successor.

Apparently, it's been too difficult to find trained C++ programmers who wish to spend time hacking Sigil.

[Hitch]

Originally Posted by eschwartz

Making epub happen: Sigil's Spiritual Successor



Apparently, it's been too difficult to find trained C++ programmers who wish to spend time hacking Sigil.

Or who were willing to take the fairly constant abuse about their "failure" to fix "flaws" that were constantly harangued for by the illiterati. I honestly believe that that, more than anything else, contributed to user_none's decision to leave it. I can't/don't speak for user_none, but that's my gut take on it after some chats with him.

Hitch