glyphIgo: font minimizer for EPUB ebooks - Page 2

Toxaris · 03-31-2013, 03:43 AM

Quote:

Originally Posted by JSWolf

Then glyphIgo needs to be fixed. ADE 2.0 and some versions between 1.7.2 and 2.0 do convert to ligatures and if glyphIgo says everything is good to go in the fonts and it's not, then there could be missing characters.

I think AlPe can decide on his own if he wants to incorporate this. I would not call it 'fixing', since it is actually an enhancement. You want to add characters to the subset which are not used within the document itself, but by the readers. So, it is not an bug but an enhancement request. I think it is rather unpolite to ask for an enhancement with the words 'needs to be fixed'.

Jellby · 03-31-2013, 03:52 AM

Quote:

Originally Posted by AlPe

Also, glyphIgo does not handle ligatures, if you mean whether it is able to detect them, collapse them and use the appropriate Unicode symbol. However, if a ligature is already specified as a single Unicode character, it is managed properly (as a single Unicode character).

Unicode characters only exist for a handful of ligatures (I mean stylistic ligatures such as "ffi", not letter-like ones like ß or æ), and they should not be used! They are only there for historical and compatibility reasons. Ligatures should be handled by the font alone.

It has already been discussed elsewhere, but it should be required for a font subsetter to properly consider ligatures, that is, at least not remove them if they are present in the original font... ideally, it should remove only unused ligatures (and "alternate" glyphs, etc., but that may be difficult to process).

Quote:

Originally Posted by JSWolf

Then glyphIgo needs to be fixed. ADE 2.0 and some versions between 1.7.2 and 2.0 do convert to ligatures and if glyphIgo says everything is good to go in the fonts and it's not, then there could be missing characters.

Missing ligatures would not result in missing characters. The only problem occurs if a font with ligatures is subset without taking them into account then at least two things could happen:

(a) The ligatures are completely removed. The original font may show, for instance, "ffi" or "Th" as ligatures, the subset font will simply show them as their individual characters, just like most renderers (which don't support ligatures) will do anyway.

(b) The subsetter is buggy, the ligatures are removed but their references are not. A renderer that does not support ligatures will not notice. A renderer that does, will show empty blocks or question marks where the ligatures would be.

But, assuming a given font is correct (i.e., it doesn't have references to non-existent characters), there's no way to know whether ligatures would have been used or not, and there's certainly nothing broken.

AlPe · 03-31-2013, 05:48 AM

@JSWolf : thanks for the lead, but I think I will not implement that, for the reasons stated by Jellby (thanks for writing them out), and because the whole tool could be made way more "precise" on more cogent levels (e.g., by implementing full EPUB parsing, in particular, style resolution and the like). I just wanted to share with the members of the public a small tool that I coded for my own EPUB reading (& authoring) needs, which grew bigger and bigger while incorporating suggestions by friends and collegues.

@Toxaris : no offence taken, I am sure JSWolf genuinely wanted to point the issue out. Moreover, I am not a native English speaker, and I know that sometimes I might sound unpolite, simply because I do not master idiomatic forms.

Jellby · 03-31-2013, 06:21 AM

Quote:

Originally Posted by AlPe

@JSWolf : thanks for the lead, but I think I will not implement that, for the reasons stated by Jellby (thanks for writing them out), and because the whole tool could be made way more "precise" on more cogent levels (e.g., by implementing full EPUB parsing, in particular, style resolution and the like).

Just be aware that if you remove all ligatures when subsetting a font, you might be actually removing the very reason why the font was embedded to start with. Think of a calligraphic font, which might have special forms for ligatures or initial or final letters, maybe it doesn't look very good without the ligatures.

AlPe · 04-02-2013, 11:18 AM

Thanks for the tip.

In the current version (1.19), glyphIgo simply retains those glyphs belonging to a given list (which is computed by filtering the source of the X(HT)ML files in the given EPUB). I am not sure what happens to the ligatures, I need to check what python-fontforge does in that case.

dgatwood · 04-03-2013, 12:54 AM

In my experience, if you remove ligatures that exist in a 'calt' table and do not remove the corresponding 'calt' table entry, most font renderers will mindlessly display a rectangular box or a space where the character should be (i.e. "b." on Jellby's list). YMMV.

In other words, if you are subsetting a font, you must do one of the following things:

Read in the 'calt' table and painstakingly go through all the glyphs to see if any character patterns match the pattern on the left (comparison) side, and if so, add the glyphs on the right (output) side to the list of glyphs to keep,
Read in the 'calt' table and keep any glyph that appears on the right (output) side no matter what, or
Strip out any 'calt' table entries where the glyph on the right (output) side is being stripped out, which in practice probably means stripping out the 'calt' table entirely.

#1 is most correct. #2 is also correct but results in slightly larger files. #3 is kind of lame, because it will probably strip out all contextual alternates, but at least it won't result in missing letters in your text.

AlPe · 04-04-2013, 02:26 PM

Wow, thanks for the detailed explanation.

Yesterday I gave a quick read at fontforge python APIs, but I failed to find a clear lead about the ligatures, when subsetting a font. I need to go through it again with greater attention, but I am quite short on time, lately.

roger64 · 07-02-2013, 11:01 AM

Hi

I was keen to test glyphIgo on LMDE 64 bits. I still have Python 2.7 but I cannot install python-htmlentitydefs and python-unicodedata using the software-manager because it did not find them.

AlPe · 07-02-2013, 11:15 AM

Sorry, I do not know how LMDE repos work.

I think that unicodedata is a core module which is automatically installed when you install Python. In Debian there is no python-htmlentitydefs package, I think it gets installed if you install BeautifulSoup (python-beautifoulsoup).

mrmikel · 07-02-2013, 01:18 PM

Quote:

Originally Posted by AlPe

Wow, thanks for the detailed explanation.

Yesterday I gave a quick read at fontforge python APIs, but I failed to find a clear lead about the ligatures, when subsetting a font. I need to go through it again with greater attention, but I am quite short on time, lately.

Based on dgatwood's description of the process, I think I would be short of time if I had a two week vacation!

AlPe · 03-08-2014, 06:07 AM

I moved glyphIgo to GitHub, and re-released it under the MIT license.

Please download the latest version from:

https://github.com/pettarin/glyphIgo

Enjoy!

04-03-2013, 12:54 AM	#21
dgatwood Curmudgeon Posts: 629 Karma: 1623086 Join Date: Jan 2012 Device: iPad, iPhone, Nook Simple Touch	In my experience, if you remove ligatures that exist in a 'calt' table and do not remove the corresponding 'calt' table entry, most font renderers will mindlessly display a rectangular box or a space where the character should be (i.e. "b." on Jellby's list). YMMV. In other words, if you are subsetting a font, you must do one of the following things: Read in the 'calt' table and painstakingly go through all the glyphs to see if any character patterns match the pattern on the left (comparison) side, and if so, add the glyphs on the right (output) side to the list of glyphs to keep, Read in the 'calt' table and keep any glyph that appears on the right (output) side no matter what, or Strip out any 'calt' table entries where the glyph on the right (output) side is being stripped out, which in practice probably means stripping out the 'calt' table entirely. #1 is most correct. #2 is also correct but results in slightly larger files. #3 is kind of lame, because it will probably strip out all contextual alternates, but at least it won't result in missing letters in your text.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
ebooks that won't let you change the font	mr ploppy	Amazon Kindle	4	10-12-2012 09:01 PM
Determine font and font size on incoming epub?	peaceridge	Calibre	4	01-30-2012 03:35 PM
Open EPUB or Adobe EPUB Library ebooks?	jana_leigh14	Kobo Tablets	2	11-08-2011 01:07 AM
Font Too Small To Read on eBooks Bought Through KoboBooks	Mysterio	Kobo Reader	13	06-29-2010 05:25 PM
Preferred Font Size in eBooks	ahi	Workshop	52	05-24-2009 12:59 PM

03-31-2013, 05:48 AM	#18
AlPe Digital Amanuensis Posts: 727 Karma: 1446357 Join Date: Dec 2011 Location: Turin, Italy Device: Several eReaders and tablets	@JSWolf : thanks for the lead, but I think I will not implement that, for the reasons stated by Jellby (thanks for writing them out), and because the whole tool could be made way more "precise" on more cogent levels (e.g., by implementing full EPUB parsing, in particular, style resolution and the like). I just wanted to share with the members of the public a small tool that I coded for my own EPUB reading (& authoring) needs, which grew bigger and bigger while incorporating suggestions by friends and collegues. @Toxaris : no offence taken, I am sure JSWolf genuinely wanted to point the issue out. Moreover, I am not a native English speaker, and I know that sometimes I might sound unpolite, simply because I do not master idiomatic forms.

04-02-2013, 11:18 AM	#20
AlPe Digital Amanuensis Posts: 727 Karma: 1446357 Join Date: Dec 2011 Location: Turin, Italy Device: Several eReaders and tablets	Thanks for the tip. In the current version (1.19), glyphIgo simply retains those glyphs belonging to a given list (which is computed by filtering the source of the X(HT)ML files in the given EPUB). I am not sure what happens to the ligatures, I need to check what python-fontforge does in that case.

04-04-2013, 02:26 PM	#22
AlPe Digital Amanuensis Posts: 727 Karma: 1446357 Join Date: Dec 2011 Location: Turin, Italy Device: Several eReaders and tablets	Wow, thanks for the detailed explanation. Yesterday I gave a quick read at fontforge python APIs, but I failed to find a clear lead about the ligatures, when subsetting a font. I need to go through it again with greater attention, but I am quite short on time, lately.

07-02-2013, 11:01 AM	#23
roger64 Wizard Posts: 2,608 Karma: 3000161 Join Date: Jan 2009 Device: Kindle PW3 (wifi)	Hi I was keen to test glyphIgo on LMDE 64 bits. I still have Python 2.7 but I cannot install python-htmlentitydefs and python-unicodedata using the software-manager because it did not find them.

07-02-2013, 11:15 AM	#24
AlPe Digital Amanuensis Posts: 727 Karma: 1446357 Join Date: Dec 2011 Location: Turin, Italy Device: Several eReaders and tablets	Sorry, I do not know how LMDE repos work. I think that unicodedata is a core module which is automatically installed when you install Python. In Debian there is no python-htmlentitydefs package, I think it gets installed if you install BeautifulSoup (python-beautifoulsoup).

03-08-2014, 06:07 AM	#26
AlPe Digital Amanuensis Posts: 727 Karma: 1446357 Join Date: Dec 2011 Location: Turin, Italy Device: Several eReaders and tablets	I moved glyphIgo to GitHub, and re-released it under the MIT license. Please download the latest version from: https://github.com/pettarin/glyphIgo Enjoy!