PDF is not an eBook format

[Ankh]

Originally Posted by ahi

Do you really consider feasible a hyphenation database that contains all actual words it addresses, along with all their compounded, conjugated/declined (and apostrophe-laden?) forms? Even for languages unlike English, where prefixes, suffices, conjugations, and declensions can create over a 100 valid and sensible words from a 3 letter root word?

Given how utterly technically simple this is, you'd think all word processors would be using it by now. (Though "present" in English, and many words in other languages would continue to be incorrectly auto-hyphenated.)

- Ahi

Feasible? Yes. Massive OED dictionaries are available for PC, this database will most likely be smaller than that.

The process would be SLOW, most likely. But it is done once, on PC, when book is made. Our desktops are very capable machines.

The situation with "present" and ambiguous cases in other languages IS NOT a concern, since we are NOT doing machine-only hyphenation here. The database can recognize such cases and ask for human intervention. You resolve it once, during book creation.

I really don't see any problems with the method.

[ahi]

Never mind. What I wrote here doesn't make sense.

[ahi]

Originally Posted by Ankh

Feasible? Yes. Massive OED dictionaries are available for PC, this database will most likely be smaller than that.

The process would be SLOW, most likely. But it is done once, on PC, when book is made. Our desktops are very capable machines.

The situation with "present" and ambiguous cases in other languages IS NOT a concern, since we are NOT doing machine-only hyphenation here. The database can recognize such cases and ask for human intervention. You resolve it once, during book creation.

I really don't see any problems with the method.

Ok... let me put aside for a moment that I don't think this idea is feasible... particularly because I might have run out of cogent arguments as to why...

How would you go about building such a database, Ankh?

Just processing oodles and oodles of PG eTexts, and manually hyphenate the words therefrom?

- Ahi

[Ankh]

Originally Posted by ahi

How would you go about building such a database, Ankh?

Just processing oodles and oodles of PG eTexts, and manually hyphenate the words therefrom?

Start with the source of nrapallo Webster 1913 dictionary.

Then yes, expect users to help with the growth of the database. The database-assisted hyphenation engine can ask for intervention whenever a word is not in the database. When job is done, process the database, extract the words that were added to basic text file, one line per hyphenated word, submit such file back to the maintainer. Review (use dictionaries and any other tools available), merge changes, new version of the database.

Open source.

[ahi]

Originally Posted by Ankh

Start with the source of nrapallo Webster 1913 dictionary.

Then yes, expect users to help with the growth of the database. The database-assisted hyphenation engine can ask for intervention whenever a word is not in the database. When job is done, process the database, extract the words that were added to basic text file, one line per hyphenated word, submit such file back to the maintainer. Review (use dictionaries and any other tools available), merge changes, new version of the database.

Open source.

I'm highly skeptical... but such a thing would be helpful to the sorts of projects I work on. Once I get my text formatting program into a bit better shape, perhaps I'll try to implement the framework for this idea.

- Ahi

[Ankh]

Originally Posted by ahi

I'm highly skeptical... but such a thing would be helpful to the sorts of projects I work on. Once I get my text formatting program into a bit better shape, perhaps I'll try to implement the framework for this idea.

- Ahi

Beware, it is a massive undertaking, especially for the database maintainer.

A rigid review policy before submission is needed, or everything can easily fall apart. In the early stage, the tool and the database will be almost useless (too many misses), but that can quickly change, since the most frequently used words and their incarnations (whatever is the reason for slightly different form) will soon find its way into the database.

A "perfect" database might never emerge, but pretty complete one (hit ratio above 99%) would be more than useful.

Your call, I am not ready to make such a commitment. Once soft hyphens are implemented on prs505, I promise that I will use the tool and contribute to the growth of the database.

[Kostas]

Originally Posted by ahi

or

λοπαδο*τεμαχο*σελαχο*γαλεο*κρανιο*λειψανο*δριμ*υπο *τριμματο*σιλφιο*καραβο*μελιτο*κατακεχυ*μενο*κιχλ* επι*κοσσυφο*φαττο*περιστερ*αλεκτρυον*οπτο*κεφαλλιο *κιγκλο*πελειο*λαγῳο*σιραιο*βαφη*τραγανο*πτερύγ ων

(Will eBook readers know how to hyphenate Greek words correctly, even if they won't concern themselves with Gikuyu?)

- Ahi

Ps.: Sorry.. not sure why the Greek is broken. All are single words, by the way... despite the forum display forcing word breaks.

Forgiven! Actually, accents are missing too...

[ahi]

Originally Posted by Kostas

Forgiven! Actually, accents are missing too...

Must be some weird unicode mangling... Oh well... Nordöstersjökustartilleriflygspaningssimulatoranlä ggningsmaterielunderhållsuppföljningssystemdiskuss ionsinläggsförberedelsearbeten made my admittedly tongue-in-cheek point well enough.

- Ahi

[Kostas]

Originally Posted by ahi

Must be some weird unicode mangling... Oh well... Nordöstersjökustartilleriflygspaningssimulatoranlä ggningsmaterielunderhållsuppföljningssystemdiskuss ionsinläggsförberedelsearbeten made my admittedly tongue-in-cheek point well enough.

- Ahi

Is it finnish?

[Elfwreck]

Originally Posted by ahi

Do you really consider feasible a hyphenation database that contains all actual words it addresses, along with all their compounded, conjugated/declined (and apostrophe-laden?) forms? Even for languages unlike English, where prefixes, suffices, conjugations, and declensions can create over a 100 valid and sensible words from a 3 letter root word?

Given how utterly technically simple this is, you'd think all word processors would be using it by now. (Though "present" in English, and many words in other languages would continue to be incorrectly auto-hyphenated.)

It's not possible to build a hyphenation database for every word, especially not every possible word in agglutinative languages, nor patchwork languages like English that create new words by smashing other words together.

However, a good, if not perfect, hyphenation algorithm could be made, based on linguistic analysis of the language. And it could be combined with a dictionary, so it would automatically put up flags for words that could be either compound words or identically-spelled words with different meanings. (I'd say "homonyms," but they might not have the same pronunciation.) It wouldn't fix all mishy-phens, but it'd allow the formatting person (whoever that is, author or editor) to quickly identify the possibilities, rather than doing a line-by-line proof every time the formatting shifts a bit.

As far as I know, "present" is always split "pre-sent," with any of its three possible pronunciations. However, unless I was using it in a sentence like
"we knew the authorization would arrive later that week, so we pre-
sent the package," I'd avoid hyphenating it to avoid confusion, because ending with "pre-" implies the long-e pronunciation.

There's no reason hyphenation software couldn't be as good as current spellcheck software--not perfect, but good enough to remove a lot of the gruntwork of proofreading, and good enough to reflow a book to avoid almost all troublesome hyphenations.

As amusing or sometimes annoying as bad hyphenations are, I'd rather publishers spent more time on actual typos, and apostrophe use, and a good table of contents. Oh, and an index for nonfiction books.

I need good content before I need great formatting.

[Abecedary]

Originally Posted by Elfwreck

As far as I know, "present" is always split "pre-sent," with any of its three possible pronunciations.

I'm generally sitting out of this discussion, but I did want to point out that Merriam-Websters breaks present as pres-ent for 3 of the 4 main usages of the word (all but the verb form).

[ahi]

Originally Posted by Kostas

Is it finnish?

Norwegian, if I recall correctly.

[jbjb]

Originally Posted by ahi

What makes you think that it's an either-or proposition? It seems a counter-intuitive assumption to me. And, to be frank, any time the publisher decides to save on getting typography or even just the hyphenation part right will not be time re-allotted to additional proofreading.

There is, I'm rather sure, no chance of improving the content accuracy of eBooks by arguing for relaxing the typographic standards thereof.

And as I've noted a good few times though, good quality typography is not difficult or time-consuming to do with the right tools and technologies, and even great typography is not prohibitively so, even for a small publisher.

What I meant was that any benefits of improved typography over what can currently be acheived on a reflowed epub or similar pale into insignificance (N.B. to me, with my particular priorities) with respect to the improvement that would result if publishers improved their basic proof reading of ebooks (and, indeed, all books). The current standard on this area is so poor, that it doesn't really strike me as at all gainful to try and convince publishers to produce hand-crafted layouts for various sizes. I'd rather concentrate on encouraging them to improve the basic accuracy of the product.

Striving for perfect (for somebody's arbitrary definition of perfect) typography on a book which is riddled with errors is simply polishing the proverbial you-know-what.

For me (and I emphasise that this is for me - as I've said, different people want different things), if the content is accurate then I'm happy with a very basic layout - minor hyphenation errors or even stacks etc. don't really bother me.

Having said that, I take your point that a format which mixed a reflowable format with specialised layouts for specific sizes would be ideal. My concern is that I can't see how publishers who can't even get the basics right are going to do it well - I'd rather they just concentrated on the fundamentals.

/JB

[ahi]

Originally Posted by jbjb

My concern is that I can't see how publishers who can't even get the basics right are going to do it well - I'd rather they just concentrated on the fundamentals.

Hmmm... and I, on the other hand, do not see publishers who are uninterested in the relatively easier work of making their book's typographic quality not plainly broken, be courageous in their undertaking of the considerably more difficult problem of general proofreading and rigorous content correction that constitute a lengthier and more arduous set of tasks.

But, to be honest, if I did think it was an either or proposition, I'd follow you in preferring them to do the proofing properly. After all, I can typeset the book myself more easily than I can proofread it.

- Ahi

[Abecedary]

This completely ignores that proofreading and typesetting duties are generally handled by entirely different people or departments, especially in an organization of, oh, say, five or more people. To me, this further emphasizes that it's not an either/or proposition.