Working on way to subset fonts for ePub/KF3

[pdurrant]

I've found a Perl-based TTF font subsetter here: https://bitbucket.org/philip/font-optimizer/overview (MIT license)

there seems to be some C++ source here: http://podofo.sourceforge.net/doc/ht...TTFSubset.html (LGPL)

Another Perl one here: http://search.cpan.org/~mhosken/Font-TTF-Scripts/ (Perl Artistic license 2.0)

A php one here: http://www.4real.gr/technical-documents-ttf-subset.html

Aha! A python one under active development as part of a PDF generation project: http://code.google.com/p/pyfpdf/ (LGPL)

I do not know if any of the projects also work on otf fonts.

matplotlib is another python project that seems to include font subsetting: https://github.com/matplotlib/matplotlib (an attribution licence)

[pdurrant]

Originally Posted by Freeshadow

Fontforge can AFAIR be controlled by python scripts.

But is a bit of overkill. I'd much rather find some self-contained python library. I've done a bit of digging and listed some projects that I've found. This is to remind me to check them out later.

[Jellby]

I wonder how those subsetters work with ligatures and glyph variants...

[pdurrant]

Originally Posted by Jellby

I wonder how those subsetters work with ligatures and glyph variants...

Those are good questions!

[JSWolf]

Originally Posted by Freeshadow

Python means fontforge could be fed with it just what I tought about.

Not a good solution as Fontforge is Linux and to get it to run on Windows, Cygwin has to be installed and Python has to be installed under Cygwin. That's a very poor solution for Windows users. We need a more universal solution.

[Doitsu]

Originally Posted by JSWolf

Not a good solution as Fontforge is Linux and to get it to run on Windows, Cygwin has to be installed and Python has to be installed under Cygwin.

You're wrong on both accounts. The Fontforge Windows build no longer requires Cygwin and Python never did.
You may want to do your own research every now and then.

[pdurrant]

In a couple of other threads, the idea has come up that it would be very useful to have a tool that could subset the fonts embedded in ePub and/or KF8 books, as that would both reduce the file sizes and make more fonts available, as some font licences require subsetting to permit embedding.

I have moved the posts from the main other thread over here.

[pdurrant]

Someone raised the question of ligatures and alternate characters. As I said, these are very good questions, as the creator of the ePub has no control over the glyph choice of the display software.

In PDFs, the subsetting task is a lot simpler, as all the glyphs used (not just characters) are fixed in the PDF.

For ePubs and KF8, I think we must take this into account in any solution. But this doesn't need to be part of the font subsetting code, which should work from a passed list of glyphs that should be included. (And should return an error if any are missing from the font.)

For ligatures we might need to generate not only a list of all characters in a file, but also of all character pairs. But, of course, there are also three character ligatures (ffi in English, for examples) and I suppose some languages might have more.

Hmmm... Perhaps we just need to include all ligatures for which the source file includes all the characters in the ligature.

Or perhaps we also need a script to get information on ligatures present in a font, so that that information can be used when parsing the XHTML.



Or should we start off with a very basic solution, and elaborate once that's working?

[Jellby]

Originally Posted by pdurrant

Hmmm... Perhaps we just need to include all ligatures for which the source file includes all the characters in the ligature.

I think that's the easiest, same with other features like kerning pairs or alternate glyphs.

[Toxaris]

The C++ class can also work with otf files according to the site. Unfortunately I cannot program in C++...

[JSWolf]

Originally Posted by Doitsu

You're wrong on both accounts. The Fontforge Windows build no longer requires Cygwin and Python never did.
You may want to do your own research every now and then.

Are you sure Python would work with that Windows compiled version of Fontforge?

[Doitsu]

Originally Posted by JSWolf

Are you sure Python would work with that Windows compiled version of Fontforge?

Unfortunately, the MinGW based FontForge Windows binary installer doesn't seem to install the fontforge Python module, but theoretically it should be possible to use Python scripts to control FontForge.

[Toxaris]

The windows binary of FontForge has more drawbacks. It crashes often.

[DiapDealer]

Because I have much more time than sense, I've done some more work on the script that counts/collects the characters used in files.

Building on the core that Man Eating Duck posted, this script will work for a single ePub, (x)html, or text file. In addition to filtering all of the html code/attributes from the results, it will also convert entities (named or otherwise) to their rendered equivalents.

It also has the ability to limit the results to a single specified CSS class (handy for determining the font-subset required for headings or drop-caps).

Python will almost always have issues printing certain unicode characters to the console on Windows OSs, so Windows users should consider just writing the results to a file and then viewing that file with an editor that supports the required character encoding.

Should work with Python 2.5 - 2.7 (maybe even earlier).

A test xhtml file is included for testing/benchmarking purposes.

Spoiler:

USAGE: characters-used.py [-h] [-o OUTFILE] [-e ENCODING] [-c CSSCLASS] FILE

This script will parse an epub/html/text file and generate a list of
unique characters used in that file.

Positional arguments:
   FILE         Input file (epub html text).

Optional arguments:
   -h, --help           show this help message and exit.
   -o OUTFILE, --outfile OUTFILE
                Output file for unique character list. (default: None)
   -e ENCODING, --encoding ENCODING
                Character encoding of input file. (default: utf-8)
   -c CSSCLASS, --cssclass CSSCLASS
                Restrict results to a specific CSS class. (default: None)

[JSWolf]

But is there a way of taking a TTF or OTF font file and subsetting it that works on Windows, OS X, & Linux without the need for Cygwin?