10-06-2012, 09:22 AM | #1 | |
temp. out of service
Posts: 2,792
Karma: 24285242
Join Date: May 2010
Location: Duisburg (DE)
Device: PB 623
|
Working on way to subset fonts for ePub/KF3
Quote:
https://www.mobileread.com/forums/sho...49#post2240749 the legal and practical need of such a script is currently discussed ibid. |
|
10-06-2012, 06:56 PM | #2 |
Resident Curmudgeon
Posts: 73,970
Karma: 128903378
Join Date: Nov 2006
Location: Roslindale, Massachusetts
Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3
|
|
10-07-2012, 09:42 AM | #3 |
Addict
Posts: 254
Karma: 69786
Join Date: May 2006
Location: Oslo, Norway
Device: Kobo Aura, Sony PRS-650
|
I believe I wrote it in php, which can be installed on Win7. It just accepts an utf-8 input text file and spits out a character list to stdout, it's not very sophisticated. I'll see if I find it when I go to work tomorrow, if not I can probably rewrite it in a few minutes if you're still interested
|
10-08-2012, 04:29 PM | #4 |
Addict
Posts: 254
Karma: 69786
Join Date: May 2006
Location: Oslo, Norway
Device: Kobo Aura, Sony PRS-650
|
@JSWolf: Seems it was Python, it works in Python 2.7. I've probably done horrible things to the Python language, but here it is, no guarantuees about anything:
Code:
import argparse, codecs parser = argparse.ArgumentParser(description='''This script will accept utf-8 text files and write a list of unique characters to stdout or an output file''') parser.add_argument("file", nargs='+',help="input (utf-8) file(s) for character counting") parser.add_argument("-o", "--outfile", help="outputfile") args = parser.parse_args() disallowed = set('') s=set() for f in args.file: s=s|set(char for line in codecs.open(f, encoding="UTF-8") for char in line if char not in disallowed) if args.outfile: print 'Writing to file: '+args.outfile; with codecs.open(args.outfile, "w", "utf-8") as f: f.write(u''.join(s)) f.close else: print u''.join(s).encode('utf-8') Use the output file option for Unicode files, as many glyhs won't show in a console. If you have questions about the code I can try to answer, but I heard there are some guys in the calibre forum who probably have a bit more experience with Python |
10-08-2012, 05:46 PM | #5 |
temp. out of service
Posts: 2,792
Karma: 24285242
Join Date: May 2010
Location: Duisburg (DE)
Device: PB 623
|
Python means fontforge could be fed with it just what I tought about.
|
10-09-2012, 08:58 AM | #6 |
Grand Sorcerer
Posts: 27,549
Karma: 193191846
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
|
It'd be nice to eliminate all characters from the script that occur inside html tags. Those wouldn't necessarily need to be a part of any embedded font since they won't be rendered.
|
10-09-2012, 09:18 AM | #7 | |
Addict
Posts: 254
Karma: 69786
Join Date: May 2006
Location: Oslo, Norway
Device: Kobo Aura, Sony PRS-650
|
Quote:
Since you might be interested only in special characters, you could just add a bunch of regular characters that you're not interested in to disallowed = set('') in line 6, ie Code:
disallowed = set('abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789') The script wasn't really intended for publication, so it's unfortunately pretty rough, and I don't really have enough experience to improve it. It works for my needs, though |
|
10-09-2012, 10:16 AM | #8 |
Grand Sorcerer
Posts: 27,549
Karma: 193191846
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
|
Spoiler:
usage: uniquechars.py [-h] [-c CODEC] [-o OUTFILE] file [file ...] An attempt to modify so that only the text of an html document is parsed and also allow the input/output of other charset encodings. The default is utf-8 if not specified on the command-line. I got it to work with either utf-8 or windows-1252. |
10-10-2012, 04:05 AM | #9 |
frumious Bandersnatch
Posts: 7,516
Karma: 18512745
Join Date: Jan 2008
Location: Spaniard in Sweden
Device: Cybook Orizon, Kobo Aura
|
I would render the HTML in a browser, copy and paste in a text file, and extract the unique chars from there.
|
10-10-2012, 06:25 AM | #10 | |
The Grand Mouse 高貴的老鼠
Posts: 71,506
Karma: 306214458
Join Date: Jul 2007
Location: Norfolk, England
Device: Kindle Voyage
|
Quote:
Now we need something that can process a TTF or OTF file and create a sub-set of the font. |
|
10-10-2012, 07:17 AM | #11 |
Grand Sorcerer
Posts: 5,584
Karma: 22735033
Join Date: Dec 2010
Device: Kindle PW2
|
BTW, Python-challenged ebook designers could simply compile an epub with KindlePreviewer/KindleGen and have a look at the detected Unicode ranges in the log file. For example, if you compile the book mentioned in roger64's post you'll see the following output:
Code:
Info(prcgen):I1045: Computing UNICODE ranges used in the book Info(prcgen):I1046: Found UNICODE range: Basic Latin [20..7E] Info(prcgen):I1046: Found UNICODE range: General Punctuation - Windows 1252 [2018..201A] Info(prcgen):I1046: Found UNICODE range: Latin-1 Supplement [A0..FF] Info(prcgen):I1046: Found UNICODE range: General Punctuation - other than Windows 1252 [2015..2017] Info(prcgen):I1046: Found UNICODE range: Latin Extended-A [100..17F] Info(prcgen):I1046: Found UNICODE range: Basic Greek [370..3FF] Info(prcgen):I1046: Found UNICODE range: Greek Extended [1F00..1FFF] |
10-10-2012, 07:24 AM | #12 |
frumious Bandersnatch
Posts: 7,516
Karma: 18512745
Join Date: Jan 2008
Location: Spaniard in Sweden
Device: Cybook Orizon, Kobo Aura
|
|
10-10-2012, 07:49 AM | #13 |
The Grand Mouse 高貴的老鼠
Posts: 71,506
Karma: 306214458
Join Date: Jul 2007
Location: Norfolk, England
Device: Kindle Voyage
|
I think that getting the ranges isn't fine grained enough. We're not wanting to check that our fonts cover the characters used, but to trim the fonts to cover only the characters used. Of course, making sure that all the needed characters are in the font will be part of this.
|
10-10-2012, 08:02 AM | #14 | |
Addict
Posts: 254
Karma: 69786
Join Date: May 2006
Location: Oslo, Norway
Device: Kobo Aura, Sony PRS-650
|
Quote:
I suspect that most methods of subsetting would also give you a "free" coverage check in the bargain. |
|
10-10-2012, 08:17 AM | #15 |
temp. out of service
Posts: 2,792
Karma: 24285242
Join Date: May 2010
Location: Duisburg (DE)
Device: PB 623
|
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
group an ARBITRARY subset of records | RotAnal | Library Management | 6 | 10-09-2012 11:53 AM |
Working with Fonts and Calibre | kiwidude | Development | 8 | 03-04-2011 07:49 PM |
Fonts not working in a converted book | snape | Sony Reader | 9 | 11-08-2010 11:46 PM |
Changing fonts not working? | tselling | Astak EZReader | 11 | 09-21-2009 03:03 PM |
Why are some fonts not working?? | daviddem | HanLin eBook | 4 | 01-22-2009 09:14 AM |