Proposal: Extending Epub with reference book tags - Page 2

Nate the great · 05-27-2009, 11:40 PM

Quote:

Originally Posted by jgray

I wasn't saying that the MS tags should be used specifically. However, since MS based their tags on TEI, I was wondering if IDPF couldn't do the same? If not TEI, then some other existing standard. Since epub is already based on existing standards, this would make more sense than starting from scratch for dictionary support.

A question occurred to me today that needs to be asked before going further. Given that TEI tags existed long before the Epub spec was finalized, why wasn't it included as a related standard? At the very least, why wasn't a subset of tags included in a manner similar to the preferred HTML vocabulary?

I wonder if there was a good reason for not using an existing standard. We may find out.

Peter Sorotokin · 05-28-2009, 12:23 AM

I do not think XDXF would work well because it is a single file. Russian-English dictionary is 100 meg XML file: loading that in handheld memory would be challenging. So, at a minimum, single XML needs to be broken into pieces. Also, it is not an issue how to represent the content: it can be done either by CSS-styled XDXF snippets or CSS-styled XHTML with classes. This part is OK, no changes to the standard are required. The issue is how to build an index that can quickly guide reading system to the appropriate part of the content. Note that a single index file won't cut it - it will likely be too large. Some sort of hierarchical structure broken between several files is needed. That, I think, is an extension to EPUB that needs to be defined (or borrowed).

jgray · 05-28-2009, 01:09 AM

Yes, the indexing system would be the most important part. I don't know how MS does their indexing, as it is internal to their Reader software. I do know that however they do it, it is very fast.

It is a shame that MS chose not to participate in IDPF. I think they could have made some useful contributions. But collaboration has never been something they were interested in.

HarryT · 05-28-2009, 03:43 AM

What is the formal procedure for proposing an extension to the ePub standard? What is the likelyhood that an extension proposed by a "member of the public", as opposed to one of the companies who are on the standard committee, will actually get adopted?

kovidgoyal · 05-28-2009, 12:58 PM

You don't really need a modification to the EPUb standard, the following should do the trick:

Split up the html containing the definitions into sub files each sub file containing only definitions for words starting with a specific set of two letters. There will be 26*26 = 676 such files. In the ncx just add navpoints for each file with a text being the two letters that the file has the words for. Then in the OPF file just add an entry indicating the EPUB is a dictionary. Now the reader software when asked for the definition of a word has to do the following:

parse 576 entries in the NCX file to find the correct html file. Parse the HTML file to find the word.

If two letters results in too large HTML files, use three letters instead.

The HTML files should be designed with minimal in file markup to speed up processing.

Nate the great · 05-28-2009, 01:21 PM

I snipped some posts and moved them over here. I had incorrect information, and took the discussion down the wrong path.

jgray · 05-28-2009, 01:25 PM

Nice idea, but we still need whatever method is used to be included in the standard, so we have interoperability. Also, a set of tags specifically for dictionary markup is needed.

Whatever tags are used and whatever indexing/lookup method is chosen probably doesn't matter too much. We just need IDPF to do something, so that reader software wil have a standard to follow.

Hey IDPF--how about letting us know if anything is being done about this.

igorsk · 05-29-2009, 09:57 AM

One thing that many dictionary formats miss is that one entry can be indexed by many headwords. This is quite important for languages like Japanese. For example, meguirau, 巡り会う, めぐり会う and めぐりあう are all different spellings of the same word and all should match the entry.

HarryT · 05-30-2009, 02:24 AM

Quote:

Originally Posted by HarryT

What is the formal procedure for proposing an extension to the ePub standard? What is the likelyhood that an extension proposed by a "member of the public", as opposed to one of the companies who are on the standard committee, will actually get adopted?

Can anybody answer these question, please? Nate?

Jellby · 05-30-2009, 04:36 AM

I don't know, but there is a forum at https://www.idpf.org/forums/ ...

jgray · 05-30-2009, 05:38 PM

Quote:

Originally Posted by Jellby

I don't know, but there is a forum at https://www.idpf.org/forums/ ...

Did you notice the last time a post was made on those forums? Also, the number of questions that were never answered? Without someone at IDPF contributing a lot more to those forums, they are dead.

Nate the great · 06-04-2009, 02:31 PM

Quote:

Originally Posted by HarryT

What is the formal procedure for proposing an extension to the ePub standard? What is the likelyhood that an extension proposed by a "member of the public", as opposed to one of the companies who are on the standard committee, will actually get adopted?

I don't know, twice. I sent an email to Michael Smith, the IDPF executive director. I have not received a response.

zelda_pinwheel · 06-04-2009, 02:35 PM

Hadrien is a member of idpf ; it's possible he could help or at least give some answer. also, garth conboy of eti is very involved and also very friendly ; you might want to contact him.

Nate the great · 06-04-2009, 03:15 PM

Quote:

Originally Posted by jgray

Nice idea, but we still need whatever method is used to be included in the standard, so we have interoperability. Also, a set of tags specifically for dictionary markup is needed.

Yes and no. Do you really think anyone will want to write code that actually makes use of all the XDXF tags? I'm not so sure. Remember, the tags aren't absolutely necessary simply to have the information. If you want the information, you can use XHTML and simply add it as text.

I do think XDXF should be considered as an extension to Epub-after it achieves 1.0 status.

If we adopt this position as part of the proposal, then the current set of tags won't need to duplicate all or even most of the abilities of XDXF. Instead, we can look at this project as a set reference tags, not dictionary tags.

BTW, the set of tags I show here are enough to provide dictionary lookup similar to Mobipocket.

jgray · 06-04-2009, 08:05 PM

I didn't say that the tags had to be XDXF. I don't really care how they are done. XHTML is fine. The point I was making was that some set of tags to markup dictionaries is needed and it needs to be standardized (by IDPF) so that the reader software can support it.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Seeking advice: My reference book	Steven Lyle Jordan	Writers' Corner	31	11-30-2009 09:49 AM
Reference Guide: How to Prepare Images for EPUB (and other) Formats	Zorba	ePub	13	11-22-2009 08:28 AM
Snipped from Proposal: Extending Epub	Nate the great	ePub	30	06-07-2009 07:32 AM
E-book for Reference	QFT	Which one should I buy?	8	10-17-2008 10:56 PM
E-books worm into reference book market	Bob Russell	News	0	09-23-2005 09:12 PM

05-28-2009, 12:23 AM	#17
Peter Sorotokin speaking for myself Posts: 139 Karma: 2166 Join Date: Feb 2008 Location: San Francisco Bay Area Device: PRS-505	I do not think XDXF would work well because it is a single file. Russian-English dictionary is 100 meg XML file: loading that in handheld memory would be challenging. So, at a minimum, single XML needs to be broken into pieces. Also, it is not an issue how to represent the content: it can be done either by CSS-styled XDXF snippets or CSS-styled XHTML with classes. This part is OK, no changes to the standard are required. The issue is how to build an index that can quickly guide reading system to the appropriate part of the content. Note that a single index file won't cut it - it will likely be too large. Some sort of hierarchical structure broken between several files is needed. That, I think, is an extension to EPUB that needs to be defined (or borrowed).

05-28-2009, 01:09 AM	#18
jgray Fanatic Posts: 553 Karma: 2928497 Join Date: Mar 2008 Device: Clara 2E & Sage	Yes, the indexing system would be the most important part. I don't know how MS does their indexing, as it is internal to their Reader software. I do know that however they do it, it is very fast. It is a shame that MS chose not to participate in IDPF. I think they could have made some useful contributions. But collaboration has never been something they were interested in.

05-28-2009, 03:43 AM	#19
HarryT eBook Enthusiast Posts: 85,544 Karma: 93383043 Join Date: Nov 2006 Location: UK Device: Kindle Oasis 2, iPad Pro 10.5", iPhone 6	What is the formal procedure for proposing an extension to the ePub standard? What is the likelyhood that an extension proposed by a "member of the public", as opposed to one of the companies who are on the standard committee, will actually get adopted?

05-28-2009, 12:58 PM	#20
kovidgoyal creator of calibre Posts: 45,145 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	You don't really need a modification to the EPUb standard, the following should do the trick: Split up the html containing the definitions into sub files each sub file containing only definitions for words starting with a specific set of two letters. There will be 26*26 = 676 such files. In the ncx just add navpoints for each file with a text being the two letters that the file has the words for. Then in the OPF file just add an entry indicating the EPUB is a dictionary. Now the reader software when asked for the definition of a word has to do the following: parse 576 entries in the NCX file to find the correct html file. Parse the HTML file to find the word. If two letters results in too large HTML files, use three letters instead. The HTML files should be designed with minimal in file markup to speed up processing.

05-28-2009, 01:21 PM	#21
Nate the great Sir Penguin of Edinburgh Posts: 12,375 Karma: 23555235 Join Date: Apr 2007 Location: DC Metro area Device: Shake a stick plus 1	I snipped some posts and moved them over here. I had incorrect information, and took the discussion down the wrong path.

05-28-2009, 01:25 PM	#22
jgray Fanatic Posts: 553 Karma: 2928497 Join Date: Mar 2008 Device: Clara 2E & Sage	Nice idea, but we still need whatever method is used to be included in the standard, so we have interoperability. Also, a set of tags specifically for dictionary markup is needed. Whatever tags are used and whatever indexing/lookup method is chosen probably doesn't matter too much. We just need IDPF to do something, so that reader software wil have a standard to follow. Hey IDPF--how about letting us know if anything is being done about this.

05-29-2009, 09:57 AM	#23
igorsk Wizard Posts: 3,442 Karma: 300001 Join Date: Sep 2006 Location: Belgium Device: PRS-500/505/700, Kindle, Cybook Gen3, Words Gear	One thing that many dictionary formats miss is that one entry can be indexed by many headwords. This is quite important for languages like Japanese. For example, meguirau, 巡り会う, めぐり会う and めぐりあう are all different spellings of the same word and all should match the entry.

05-30-2009, 04:36 AM	#25
Jellby frumious Bandersnatch Posts: 7,543 Karma: 19001583 Join Date: Jan 2008 Location: Spaniard in Sweden Device: Cybook Orizon, Kobo Aura	I don't know, but there is a forum at https://www.idpf.org/forums/ ...

06-04-2009, 02:35 PM	#28
zelda_pinwheel zeldinha zippy zeldissima Posts: 27,827 Karma: 921169 Join Date: Dec 2007 Location: Paris, France Device: eb1150 & is that a nook in her pocket, or she just happy to see you?	Hadrien is a member of idpf ; it's possible he could help or at least give some answer. also, garth conboy of eti is very involved and also very friendly ; you might want to contact him.

06-04-2009, 08:05 PM	#30
jgray Fanatic Posts: 553 Karma: 2928497 Join Date: Mar 2008 Device: Clara 2E & Sage	I didn't say that the tags had to be XDXF. I don't really care how they are done. XHTML is fine. The point I was making was that some set of tags to markup dictionaries is needed and it needs to be standardized (by IDPF) so that the reader software can support it.

Advert

Advert