e-Book Tools - a Word add-in

[Toxaris]

I got a question today about Mac support for this add-in. The question was if Word 2016 for Mac could be supported. I will also place my answer here.

This add-in uses VSTO and produces COM based add-ins. Those are not supported on Mac and, as far as I know and understood, Microsoft will not create the libraries to enable that.
Now, that being said, as of Word 2016 there is a new manner of creating add-ins. Those can run on all types of Word 2016 (web, desktop) for both Windows and Mac. There are however a number of issues with that:

* No backwards compatibility, it will only work for Word 2016. This is a big issue!
* As it is relatively new, documentation and experience is scarce
* No idea yet if this will catch on, it could be phased out quickly
* It would mean a complete rewrite of the add-in, as it is based on web-technologies. Nothing can be reused as of now, I hope this will change in the future when this new method is more mature.
* It would require the need of a web-server to host the add-in, for example on Azure. That would effectively mean that the add-in would no longer be free to use. That is not what I want.

So, in short. No add-in for Mac, also not in Word 2016.

[Tex2002ans]

I was recently discussing EPUB Tools and the Symbol Font -> Unicode functionality.

It made me think of Webdings + Wingdings + Wingdings 2/3. When I tested with those four fonts, EPUB Tools exported the characters as your basic "abc" instead of converting to the Unicode equivalent glyphs:

• Normal Text
  
  • a
    
  • b
    
  • c
    
• Webdings
  
  • Heavy Check Mark: ✔
    
  • Bicycle: 🚲
    
  • White Large Square: ⬜
    
• Wingdings
  
  • Cancer: ♋
    
  • Leo: ♌
    
  • Virgo: ♍
    
• Wingdings 2
  
  • Heavy North East Pointing Bud: 🙦
    
  • Heavy North West Pointing Bud: 🙤
    
  • Heavy South West Pointing Bud: 🙥
    
• Wingdings 3
  
  • Right-Shaded White Rightwards Arrow: ➩
    
  • Leftwards Back-Tilted Shadowed White Arrow: 🢨
    
  • Back-Tilted Shadowed White Rightwards Arrow: ➫
    

Here is the Unicode 7.0 documents when they introduced all of the glyphs:

http://std.dkuug.dk/jtc1/sc2/wg2/docs/n4022.pdf
http://std.dkuug.dk/jtc1/sc2/wg2/docs/n4115.pdf
http://std.dkuug.dk/jtc1/sc2/wg2/docs/n4239.pdf

These have tables listing the Wingdings/Webdings IDs -> Unicode #.

I also found this site which lists the HTML equivalent for Wingdings + Wingdings 2:

http://www.alanwood.net/demos/wingdings.html
http://www.alanwood.net/demos/wingdings-2.html

although that HTML site seems to be be dated. He says many of the Wingdings don't have an equivalent UTF-8... even though they are all spelled out in the documentation (as far as I can see).

Side Note: I was also able to use BabelMap to generate a giant list of characters added in Unicode 7.0 (see attachment) + the Symbola font here:

http://users.teilar.gr/~g1951d/

Maybe that might help in some way.

Side Note #2: Maybe you could subset/embed those symbols if needed. Many of those more obscure symbols don't even appear in most fonts (many don't even appear for me in Firefox. I shudder to think of their support in most ereaders). But converting them to actual UTF-8 will be infinitely more useful than the current "abc"s.

[Toxaris]

At the moment the font's are ignored, so I am not surprised by this. Although, there is a setting in which the used fonts are referenced and, if possible, embedded in the ePUB. It should do the trick I think. That being said, embedding is usually not allowed for most commercial fonts, that would include webdings.
One of the issues is that the support for these characters in the default fonts on the readers are probably non-existing. So the references would be there, but rendered as a question mark or box usually.

That all being said, I do see added value in this. In fact, at the moment I am working on a grand overhaul of the whole 'convert to html' part. It is being rewritten from the ground up using totally different techniques and methods. It should be a lot faster and better portable to other applications.
This topic actually came up in my notes several times (both fonts specific and symbols specifically), so it is something I am addressing there. The prep-work you did will probably help me a lot, so thanks!
The new library that I am building will feature support for both fonts and symbols in some form. It is nowhere near from finished yet, as I want to have at least support the same Word functions as I do now. The basics work, but currently I am working on more complex tables (row and column spans) and lists.
Work is slow, especially since my laptop decided to end his life after four years and I have no replacement.

[BetterRed]

Originally Posted by Toxaris

Work is slow, especially since my laptop decided to end his life after four years and I have no replacement.

I'm happy make a modest contribution to a - NewLappyForTox - fund if someone gets one going.

BR

[Tex2002ans]

Originally Posted by Toxaris

At the moment the font's are ignored, so I am not surprised by this. Although, there is a setting in which the used fonts are referenced and, if possible, embedded in the ePUB. It should do the trick I think. That being said, embedding is usually not allowed for most commercial fonts, that would include webdings.

In the case of Webdings/Wingdings, they are non-Unicode fonts. They remap the characters -> completely different symbols. You wouldn't want to embed that garbage into an EPUB anyway (think of all the Text-to-Speech disasters it would cause)!

Also, it seems like Unicode already did all the hard work by absorbing those 4 dingbat fonts completely.

Side Note: From what I could tell, the guy who created Symbola released it for anyone to use for any use. Straight from his site:

"In lieu of a licence; fonts and documents in this site are free for any use".

According to fileformat.info, it is one of the very few fonts that actually has a bunch of the more obscure symbols like:

No Piracy: http://www.fileformat.info/info/unic...f572/index.htm
Right Speech Bubble: http://www.fileformat.info/info/unic...f5e9/index.htm
Capitulum: http://www.fileformat.info/info/unic...2e3f/index.htm

Side Note #2: Here is the topic that spawned this discussion, "Problem with inline image":

https://www.mobileread.com/forums/sho...d.php?t=278187

eggheadbooks1 was working on a book with a Musical Note in Webdings:

• Normal Text
  
  • ¯
• Webdings
  
  • Beamed Eighth Notes: ♫

I was just saying to use the UTF-8 and embed a font, which lead me down this whole researching rabbit hole.

Originally Posted by Toxaris

One of the issues is that the support for these characters in the default fonts on the readers are probably non-existing. So the references would be there, but rendered as a question mark or box usually.

I would still say this would be better than the current "abc", or some sort of image embedding... and Unicode support is only going to go up from here.

I suspect more common symbols like:

Lock: http://www.fileformat.info/info/unic...F512/index.htm
Open Lock: http://www.fileformat.info/info/unic...F513/index.htm

are going to be added into fonts in phones, and that stuff will propagate into usage elsewhere. Next thing you know, all those new amazing Unicode smilies are going to be creeping into books too! Might as well stay ahead of the curve!

[DaleDe]

There is quite a bit of this information in our wiki, special characters, HTML 5 Named character references, and other places. Of course more complete data could always be added there for reference. Anyone can add to our wiki to make it more useful to others.

Dale

[Toxaris]

The symbol font is usually not the issue anyway, those should render allright already. I have thought a bit more about this and did some prepwork (slow day at work). I will use the unicode characters, but with a warning about characters that might not be supported. I will also put in a reference to the symbola font since it seems complete.

[Toxaris]

In the new HTML engine there will be a conversion from the Symbol/Webdings/Wingdings/Wingdings2/Wingdings3 glyph(s) to their corresponding Unicode glyph, even for the most obscure. I made the mapping (the links helped but were nowhere near complete) and the tests work.

On my site I will place a page that shows the complete mapping for all of these 5 symbol fonts to their unicode glyphs (including hex and decimal code) for reference. I hope to do that today.

[Tex2002ans]

So another potential enhancement that I just ran into today.

1. (See attached EPUB) Create an EPUB that was given metadata:

"dc:language en-GB"

and this HTML:

  <p>colour rumour programme</p>

  <p><span lang="en-US" xml:lang="en-US">color rumor program</span></p>

2. Import EPUB using EPUB Tools.

3. The DOCX language is set to (I assume the user's default language?).

In my case, in Microsoft Word, all of the text is marked as "English (U.S.)", so words like "colour" + "rumour" + "programme" are marked as "misspelled"... even though it SHOULD be a British-spelled book.

Expected Functionality

Import EPUB should look at the EPUB Language Metadata, and make the DOCX Language = EPUB Language.

Potential Side Functionality

On Import it could transfer the HTML lang + xml:lang as well? Maybe this could be an option buried in the Preferences (default to Off).

That would be helpful for properly marked up multi-language EPUBs.

Originally Posted by Toxaris

In the new HTML engine there will be a conversion from the Symbol/Webdings/Wingdings/Wingdings2/Wingdings3 glyph(s) to their corresponding Unicode glyph, even for the most obscure.

Glorious!

Originally Posted by Toxaris

On my site I will place a page that shows the complete mapping for all of these 5 symbol fonts to their unicode glyphs (including hex and decimal code) for reference. I hope to do that today.

Definitely link it when you complete it. I didn't have time to look much further since I made the post here, but thanks for saving me all that legwork.

[Toxaris]

Originally Posted by Tex2002ans

So another potential enhancement that I just ran into today.

1. (See attached EPUB) Create an EPUB that was given metadata:

"dc:language en-GB"

and this HTML:

  <p>colour rumour programme</p>

  <p><span lang="en-US" xml:lang="en-US">color rumor program</span></p>

2. Import EPUB using EPUB Tools.

3. The DOCX language is set to (I assume the user's default language?).

In my case, in Microsoft Word, all of the text is marked as "English (U.S.)", so words like "colour" + "rumour" + "programme" are marked as "misspelled"... even though it SHOULD be a British-spelled book.

Expected Functionality

Import EPUB should look at the EPUB Language Metadata, and make the DOCX Language = EPUB Language.

Potential Side Functionality

On Import it could transfer the HTML lang + xml:lang as well? Maybe this could be an option buried in the Preferences (default to Off).

That would be helpful for properly marked up multi-language EPUBs.

Of course it is more complex than that... From the top of my head the span tag with the language should be imported correctly. I do not think it gets scrubbed from the source. I will have to check in the source for that, but I am quite certain of that.
There are two situations where the language setting is changed:
1. When the preparation/preprocess procedure is run AND this is specified in the settings
2. Upon running the alternative spelling control. The language of the setting in step 1 is used. The alternative spelling control is not multilingual for several reasons (performance a very big one!)

Described above is the designed situation. Of course, if no language is specified in the imported HTML, Word will use its general setting.

A special mention for the settings for the ePUB export. The generic language for the ePUB can be specified there.
During the export to HTML (and thus ePUB) the language setting for specific words/sentences are ignored I think. I would have to check to be certain, but I am pretty sure.
There is hope though! With the new HTML conversion engine this would no longer be the case. It will be an option, but the language settings of specific words/sentences will be retained there during the export. It is one more reason why I am working on this.

Originally Posted by Tex2002ans

Definitely link it when you complete it. I didn't have time to look much further since I made the post here, but thanks for saving me all that legwork.

I will. I forgot to do it yesterday...

[Toxaris]

The mapping of the 5 symbol fonts to their unicode counterparts can be found here: http://www.toxaris.nl/glyphs/symbols.html

[DaleDe]

Originally Posted by Toxaris

The mapping of the 5 symbol fonts to their unicode counterparts can be found here: http://www.toxaris.nl/glyphs/symbols.html

Thank you very much. I have placed links to your stuff in our wiki. I also built a page for your Word add-in. You are welcome to flush it out if you wish.

Dale

[Tex2002ans]

Originally Posted by Toxaris

The mapping of the 5 symbol fonts to their unicode counterparts can be found here: http://www.toxaris.nl/glyphs/symbols.html

One quick typo:

"Wingding 3" -> "Wingdings 3"

[Toxaris]

Originally Posted by Tex2002ans

One quick typo:

"Wingding 3" -> "Wingdings 3"

I will correct it.

[Toxaris]

Originally Posted by DaleDe

Thank you very much. I have placed links to your stuff in our wiki. I also built a page for your Word add-in. You are welcome to flush it out if you wish.

Dale

Thanks, I will check it out.