Sigil "Preserve Entities" is not working for me

GrannyGrump · 09-01-2015, 08:17 AM

I use a lot of custom fonts, with the glyphs assigned/mapped to the Symbol Character Set ("private use area") --- dingbats and other symbols. These have entity names such as  the numeric equivalent for that one is 61472;.

No matter how I enter these in the Preserve Entities list (with the xF0 code, or the numeric code), Sigil refuses to preserve them. I have Clean Source settings set to none. When I save the epub(s), everything looks fine. But when I re-open the file, those entities have been converted to "characters."

Since these are non-standard glyphs, in code view the only thing that displays is the "missing glyph" empty box. This makes search and replace more difficult than it has to be.

What, if anything, am I doing wrong? Is it because Sigil can't recognize these entities due to them being in the private use area?

DiapDealer · 09-02-2015, 01:01 PM

Just so you know, I can verify that you're not doing anything wrong. It just doesn't seem to work for all entities. It's not limited to private use entities, I don't think. I tried it with the hex entities for the standard unicode dingbat symbols, and the same thing happened (converted to characters upon re-opening). I haven't looked at the code behind it yet, but it seems as if something in the opening vetting/well-formed checks might be bypassing the Preserve Entities logic somehow.

KevinH · 09-02-2015, 01:16 PM

Hi,
Are the non-wrking entities missing from this list:

https://github.com/Sigil-Ebook/Sigil...MLEntities.cpp

These are the recognized entities that PreserveEntities can work with. This list is by no means complete. Please note that Qt BookView/Preview and even html5 aware parsers like gumbo literally remove all entities actually replacing them with the proper unicode character. Then upon return PreserveEntities simply looks up the unicode code values given back returned source code, and replaces them with the allowed entity codes in the list.

So perhaps, either the entities are not recognized by Qt widgets and or parsers as legal unicode characters and so are not properly converted to a specific single unicode value. If that is the case, PreserveEntities can do nothing to put them back in your code. I will take a closer look at the code to see if I can pass though all numeric entities (as opposed to named ones) and get them back.

Hope this explains things a bit better.

KevinH

GrannyGrump · 09-03-2015, 04:32 AM

@Diap --- good to know I wasn't messing up, but I would rather have had a fixable human error so it would work for me.

@KevinH --- ya, your list seems to top out at 4-digit codes, and are for "standard" Unicode symbol characters. The codes I work with have 5 digits, so I don't know if that is anything that can be fixed.

I had also hoped that it might be my particular install (I use the portable version from PortableApps.com), but DiapDealer's reply implies that is not the case.

Thanks, hope you can tweak it!
But I still love Sigil, and they would have to tear it from my dead and bleeding computer!!!

09-01-2015, 08:17 AM	#1
GrannyGrump Obsessively Dedicated... Posts: 3,200 Karma: 34977896 Join Date: May 2011 Location: JAPAN (US expatriate) Device: Sony PRS-T2, ADE on PC	Sigil "Preserve Entities" is not working for me I use a lot of custom fonts, with the glyphs assigned/mapped to the Symbol Character Set ("private use area") --- dingbats and other symbols. These have entity names such as  the numeric equivalent for that one is 61472;. No matter how I enter these in the Preserve Entities list (with the xF0 code, or the numeric code), Sigil refuses to preserve them. I have Clean Source settings set to none. When I save the epub(s), everything looks fine. But when I re-open the file, those entities have been converted to "characters." Since these are non-standard glyphs, in code view the only thing that displays is the "missing glyph" empty box. This makes search and replace more difficult than it has to be. What, if anything, am I doing wrong? Is it because Sigil can't recognize these entities due to them being in the private use area? Last edited by GrannyGrump; 09-01-2015 at 08:21 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
"Doghouse Diaries" and "PHD Comics" not working	frakman1	Recipes	0	08-22-2014 11:32 AM
Installing Sigil. "Associate ebooks with Sigil"	automa	Sigil	2	06-07-2013 08:29 AM
Can I preserve entities when converting from html? (To avoid unicode on kindle)	krunkster	Calibre	1	04-07-2009 05:11 PM

09-02-2015, 01:01 PM	#2
DiapDealer Grand Sorcerer Posts: 27,549 Karma: 193191846 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	Just so you know, I can verify that you're not doing anything wrong. It just doesn't seem to work for all entities. It's not limited to private use entities, I don't think. I tried it with the hex entities for the standard unicode dingbat symbols, and the same thing happened (converted to characters upon re-opening). I haven't looked at the code behind it yet, but it seems as if something in the opening vetting/well-formed checks might be bypassing the Preserve Entities logic somehow.

09-02-2015, 01:16 PM	#3
KevinH Sigil Developer Posts: 7,644 Karma: 5433388 Join Date: Nov 2009 Device: many	Hi, Are the non-wrking entities missing from this list: https://github.com/Sigil-Ebook/Sigil...MLEntities.cpp These are the recognized entities that PreserveEntities can work with. This list is by no means complete. Please note that Qt BookView/Preview and even html5 aware parsers like gumbo literally remove all entities actually replacing them with the proper unicode character. Then upon return PreserveEntities simply looks up the unicode code values given back returned source code, and replaces them with the allowed entity codes in the list. So perhaps, either the entities are not recognized by Qt widgets and or parsers as legal unicode characters and so are not properly converted to a specific single unicode value. If that is the case, PreserveEntities can do nothing to put them back in your code. I will take a closer look at the code to see if I can pass though all numeric entities (as opposed to named ones) and get them back. Hope this explains things a bit better. KevinH

09-03-2015, 04:32 AM	#4
GrannyGrump Obsessively Dedicated... Posts: 3,200 Karma: 34977896 Join Date: May 2011 Location: JAPAN (US expatriate) Device: Sony PRS-T2, ADE on PC	@Diap --- good to know I wasn't messing up, but I would rather have had a fixable human error so it would work for me. @KevinH --- ya, your list seems to top out at 4-digit codes, and are for "standard" Unicode symbol characters. The codes I work with have 5 digits, so I don't know if that is anything that can be fixed. I had also hoped that it might be my particular install (I use the portable version from PortableApps.com), but DiapDealer's reply implies that is not the case. Thanks, hope you can tweak it! But I still love Sigil, and they would have to tear it from my dead and bleeding computer!!!

Advert