LaTeX Source Packages & Autogeneration

[ahi]

Originally Posted by frabjous

Anything that uses e-Ink technology still uses "pages" -- we press a button and get a completely different stock of words. It's not a "scroll" like with a browser. For most purposes, I'm happier with pages.

And, lest my earlier pointing out is missed by some:

Scrolls use pages:

http://en.wikipedia.org/wiki/Scroll

A scroll is usually divided up into pages, which are sometimes separate sheets of papyrus or parchment glued together at the edges, or may be marked divisions of a continuous roll of writing material. The scroll is usually unrolled so that one page is exposed at a time, for writing or reading, with the remaining pages rolled up to the left and right of the visible page. It is unrolled from side to side, and the text is written in lines from the top to the bottom of the page.

It therefore seems a rather modern fancy that pages are somehow unnecessary, or downright negative.

- Ahi

[kovidgoyal]

Wow what you've described seems to require an awful lot of labor on the part of e-book creators, but I guess if you want a layout by hand, then an awful lot of labor is what you're in for.

Go for it, I'll be interested to see what you can come up with.

[acidzebra]

Originally Posted by kovidgoyal

Wow what you've described seems to require an awful lot of labor on the part of e-book creators, but I guess if you want a layout by hand, then an awful lot of labor is what you're in for.
Go for it, I'll be interested to see what you can come up with.

If you have somewhat decent source material, it could be as simple as
source (pref. HTML or RTF) > pacify.py > TeX document > PDF with little or no manual intervention. For perfectionists like the typographers here this will never be an option but for us joe schmoe users it would definitely yield a vastly better-looking book to read with little extra work.

I've been toying with this process for the past week and the custom 505-formatted PDFs with full justification and proper hyphenation, my fave font and so on (all this from a properly chosen /preamble and document layout) are really worth it IMHO.

[ahi]

Originally Posted by kovidgoyal

Wow what you've described seems to require an awful lot of labor on the part of e-book creators, but I guess if you want a layout by hand, then an awful lot of labor is what you're in for.

Go for it, I'll be interested to see what you can come up with.

I'm not sure I agree with the "an awful lot of labor" part. But, certainly, the more time the eBook creator spends the better the output will be.

Think of it as starting out with a LaTeX source code that can generate a reasonably good (probably via \sloppy and other such relaxations of strictness) document for any size... then you start looking at the output at the sizes you actually expect people to generate (5" page size, 6" page size, with 11pt and 14pt fonts--4 different versions, basically) and you eliminate any blatant errors. (e.g.: a quoted letter's signature line ending up a widow, quoted poetry being poorly line-broken due to an overly narrow page-size, et cetera)

As oft-noted before, it took me a bit less than a full day's of work to generate 8 different versions of my The Art of War eBook... and frabjous is of the opinion that with the right set of commands/macros and some practice, it need not even take that long.

Originally Posted by acidzebra

If you have somewhat decent source material, it could be as simple as
source (pref. HTML or RTF) > pacify.py > TeX document > PDF with little or no manual intervention. For perfectionists like the typographers here this will never be an option but for us joe schmoe users it would definitely yield a vastly better-looking book to read with little extra work.

I've been toying with this process for the past week and the custom 505-formatted PDFs with full justification and proper hyphenation, my fave font and so on (all this from a properly chosen /preamble and document layout) are really worth it IMHO.

That's my ultimate goal. And it could literally be that simple, with a clever interactive mode where the script's assumptions about autodetected chapter titles, poetry, et cetera can be confirmed or disabused by the user *before* the output is written. (i.e.: you might not need to adjust/fix chapters and/or poems/quoted letters at all)

What the eBook maker might additionally do in the spirit of getting multiple fixed layout versions figured out for users in advance is... well, what I describe above in my answer to Kovid.

- Ahi

[ahi]

Maybe I'll make a merging of Adam's Diary and Eve's Diary the sample eBook I'll play around with.

- Ahi

[frabjous]

Originally Posted by ahi

As oft-noted before, it took me a bit less than a full day's of work to generate 8 different versions of my The Art of War eBook... and frabjous is of the opinion that with the right set of commands/macros and some practice, it need not even take that long.

It took me significantly less than a day's work to modify my source for Russell's Introduction to Mathematical Philosophy (link above) to make it available in six different sizes, even using existing tools, though I wasn't watching the clock. (Maybe 3-4 hours?)

Still, to get "arbitrary" sizing would be much more difficult. But again, this was my first attempt, and if the matter becomes routine, and we encourage people to actually develop new LaTeX packages especially for the task, then who knows how little time it might take? (Of course, my expectations are probably unreasonable high, but I think it's best to shoot for the moon, at least until you fail and know why you can't...)

[ahi]

Originally Posted by frabjous

Well, to be fair, my thought was that after tools such as the one we're hypothesizing are created it wouldn't take very long for publishers to offer multiple sizes. Kovid is referring to the time to actually create the tool, which is a little less clear.

I thought Kovid thought the work required even with the use of such tools would be excessive.

Originally Posted by frabjous

Athough it took me significantly less than a day's work to modify my source for Russell's Introduction to Mathematical Philosophy (link above) to make it available in six different sizes, even using existing tools.

There you go.

Truth be told, I'm sure the amount of manual attention an eBook should ideally receive (in preparation for multi-layout) will differ greatly based on the content and type of content... but yes, I think 7 hours (8 hour work day - 0.5 hour lunch - 2 * 0.25 hour breaks) is more than sufficient to take an average eBook's LaTeX source and customize it to look better (than the automation managed by itself) for a variety of page size and font size combinations.

And, frankly, if 7 hours is too much time for a publisher to put into preparing an eBook... chances are it will show, even in the ePub edition.

Originally Posted by frabjous

Still, to get "arbitrary" sizing would be much more difficult. But again, this was my first attempt, and if the matter becomes routine, and we encourage people to actually develop new LaTeX packages especially for the task, then who knows how little time it might take? (Of course, my expectations are probably unreasonable high, but I think it's best to shoot for the moon, at least until you fail and know why you can't...)

I think the key is that "arbitrary" sizing isn't.

1) Take the smallest screen size people are likely to read an eBook on (some ludicrous mini cellphone's tiny as hell screen, presumably) and put in conditional commands to relax LaTeX strictness sufficiently for screen sizes ranging from that to maybe 3" so that the result is readable.

2) Get a slightly less strictness-relaxed conditional command in there for sizes 3" - 6".

3) Get a slightly less strictness-relaxed conditional command in there for sizes 6"+.

4) Focus on specific screen-size/font combinations you expect people to use. Probably 5"/11pt, 6"/11pt, 5",14pt, and 6"/14pt are a good start. Eliminate blatant/obvious errors.

5) Perhaps put some conditional commands in there to do multi-column for screen-sizes larger than 9", if your text is such that it would benefit from that.

And the result should be a LaTeX source that can generate PDF files that look at least as good, and probably better, than alternative formats on devices ranging from tiny cellphone screens, popular 5" and 6" eBook readers, and larger screened premium eBook readers too. Not the same PDF, of course... and the ones you gave personal attention to will definitely look better. But basically the above should cover pretty much everything of relevance... in not too many steps.

- Ahi

[frabjous]

Originally Posted by ahi

I thought Kovid thought the work required even with the use of such tools would be excessive.

Indeed. That's why I edited my post, since I reread what he wrote, and you're right.

4) Focus on specific screen-size/font combinations you expect people to use. Probably 5"/11pt, 6"/11pt, 5",14pt, and 6"/14pt are a good start. Eliminate blatant/obvious errors.

I think it's a very good idea to start identifying what we think those would be. Those are probably a good start, but the iPhone screen is probably another one (50mm x 75mm, which is unfortunately not just a different size, but different ratio from the 90mm x 120mm of a 6" e-Ink.

I've never seen the exact specs on the 5" screens: they're at least the same ratio, right?

I also agree that what we're after is not going to be, at least not for a long time, equal to what you'd get giving individual attention to each one, but still ought to be noticeably better than what we're getting with, e.g., the three ePub zoom factors on a Sony PRS-505...

[Jellby]

For those devices that support screen rotation (Opus, iPhone...) are portrait and landscape to be considered two different "page sizes"?

[ahi]

Originally Posted by Jellby

For those devices that support screen rotation (Opus, iPhone...) are portrait and landscape to be considered two different "page sizes"?

It's probably simplest to do so, yes.

90mm x 120mm is one page size (portrait)
120mm x 90mm is another (landscape)

The fact that they have the same area is of no consequence, I think.

I imagine it would only make sense to give special attention to landscape page sizes for larger screen devices... or if the content called for it for some special reason.

Originally Posted by frabjous

I think it's a very good idea to start identifying what we think those would be. Those are probably a good start, but the iPhone screen is probably another one (50mm x 75mm, which is unfortunately not just a different size, but different ratio from the 90mm x 120mm of a 6" e-Ink.

Would the ratio difference really matter? Minimal resizing of PDF pages ought to be of fairly little consequence/impact to the displayed page.

Originally Posted by frabjous

I also agree that what we're after is not going to be, at least not for a long time, equal to what you'd get giving individual attention to each one, but still ought to be noticeably better than what we're getting with, e.g., the three ePub zoom factors on a Sony PRS-505...

Well... though I am pro-individual-attention, I will readily admit that even blunt fixes like \sloppy and your own similar suggestion (that I cannot presently recall) still result in reasonably good looking output. So less than perfect probably will still translate to "pretty good" in most instances.

- Ahi

[frabjous]

To avoid duplicating the work of others, it looks like there's some useful information here from Hadrien from Feedbooks, who generates ebooks with LaTeX. It appears he uses a modified version of this html2latex tool for part of the task, so it might be a place to start.

[ahi]

Originally Posted by frabjous

To avoid duplicating the work of others, it looks like there's some useful information here from Hadrien from Feedbooks, who generates ebooks with LaTeX. It appears he uses a modified version of this html2latex tool for part of the task, so it might be a place to start.

He doesn't anymore, but I'll definitely check those links out.

I'd also be curious what you think about the approach for pacify discussed between me and ekaser (in the pacify thread, obviously).

- Ahi

[frabjous]

Interesting. Actually, I wish I had known about that Prince XML software sooner. That's apparently what is being used for the "Custom PDF" option on feedbooks, which I've just now tried, and am getting pretty good results.

My main complaint is that the one I tried did not have "smart quotes", but straight ones, but I take it that's a problem with the Gutenberg source.

Here we have a working model of almost precisely what we're after, so it's worth poking into a bit... too bad Prince XML isn't open source.

Funny thing is that I could swear the chapter titles are generated with LaTeX's fncychap package... maybe someone has ported it...

[ahi]

Originally Posted by frabjous

Interesting. Actually, I wish I had known about that Prince XML software sooner. That's apparently what is being used for the "Custom PDF" option on feedbooks, which I've just now tried, and am getting pretty good results.

My main complaint is that the one I tried did not have "smart quotes", but straight ones, but I take it that's a problem with the Gutenberg source.

Here we have a working model of almost precisely what we're after, so it's worth poking into a bit... too bad Prince XML isn't open source.

Funny thing is that I could swear the chapter titles are generated with LaTeX's fncychap package... maybe someone has ported it...

I prefer feedbooks to any other similar page out there... but find their stuff decidedly lower quality than what I can generate via LaTeX. Though I am yet to consider the matter with cold rationality... so I cannot say how much is in my head, and how much is based on undeniable differences.

And yeah, the lack of smart quotes is definitely a task for preprocessing.

I guess pacify (and/or other scripts/programs like it) might become a #0 in the thread-starting triumvirate?

- Ahi

[frabjous]

Well, I agree that feedbooks isn't giving you the same quality, but it doesn't offer nearly as many options as Prince allows. I expect there are some things, like math, it still won't do as well, since I don't see any sign of MathML (or similar) support, but I'll still be interested in what else it can do. For very simple, text-only, books, it may do as well as the other options we're considering, unless you really wow us with pacify when you get around to (X)HTML input...

Since I have HTML source of the Russell book as well, whose CSS I wrote and can edit, I'll try making a PDF with it using Prince when I get a chance and do some comparisons.

I think the hyphenation algorithm is the same as TeX's.

Unfortunately, grumble grumble, it wouldn't be as easy to install Prince on this computer as on my home computer, since they only offer a debian package for 32-bit linux, and I'm using 64-bit Ubuntu at the moment...