Atlantic monthly: why few 20th century books are available as ebooks

[Barty]

http://www.theatlantic.com/business/...ebooks/284486/

A crucial factor seems to be court ruling that says ebooks require separate licensing and the original agreement does not apply to electronic or other format.

The article does not discuss demand. Of course, how much demand is there for books pre 1923, but they are widely available both for free and sale.

PS: A similar legal snag keeps many movies and tv shows from being released on discs. The non original songs used on soundtracks are often licensed only for the movie release. The movie studios prefer to pay nothing, whereas the song copyright holders want to extract as much as they can get. For small movies or documentaries, it's often not worth it for the studios. (Mad Men reportedly paid $250k for using 20secs of a Beatles song. If that covers DVD and streaming, it's a pretty good deal.)

[SteveEisenberg]

The legal factor mentioned in the OP article is big when the author is hard to locate or deceased. But I think the OP article underestimates the work involved in proofreading a scanned book.

Here's a bestselling no-eBook-available 1994 book with a living, New York City-residing, author who still has (allowing for mergers) the same publisher today:

http://www.amazon.com/Power-Broker-R...s=robert+moses

Author Robert Caro's newer books are eBooks, but his first is not. While there could be some complicated explanation in this case, I think the most likely reason for much I want to read being paper-only is that correcting misscans is expensive.

[bgalbrecht]

Is the paper book out of print also? My suspicion is that between the legal costs of acquiring the book and any advances on royalties may be greater than the cost of outsourced ebook conversions. Either way, the publishers have to decide whether they expect to make enough sales to make it profitable. A lot of authors who are self-publishing their backlist probably have lower overhead and can tolerate a slower payback.

[fjtorres]

Originally Posted by SteveEisenberg

But I think the OP article underestimates the work involved in proofreading a scanned book.

Nope.
With modern professional-grade scanners and software it is relatively fast and easy.

I've done technical papers at work and narrative fiction and non-fiction for personal use. A 400 page book can be scanned in under an hour, ocr'ed in 30 minutes, and proofed in 3-4 hours and no special skill is required; it's the kind of work you can give to a summer intern.

With the proper tools and workflow the proofing is reduced primarily to looking for broken sentences as the ocr software is effectively perfect when fed 400-dpi grayscale images. For non-fiction, the biggest effort is tagging the footnote/endnote hyperlinks.

In our case, the proper tools were an Epson GT-series scanner (good for up to 100 pages a minute through the sheet feeder or 8 pages a minute in manual mode), Omnipage OCR (Abbey was a bit better for the technical papers, but Omnipage produced better MS Word docs), and Presto Pagemanager Pro. Total cost is about $2000 but as a one-time expense it is easily justified in a corporate environment.

Old Technical papers are a lot more complex because of the charts and tables often need manual cleaning to bring up to modern standards but our in-house graphics specialist could process one in a couple of hours when we were converting our archives to all digital ten years ago. It helped that our management was technically literate and believed in investing in productivity tools.

It's not rocket science or brain surgery.

[fjtorres]

Originally Posted by bgalbrecht

My suspicion is that between the legal costs of acquiring the book and any advances on royalties may be greater than the cost of outsourced ebook conversions. Either way, the publishers have to decide whether they expect to make enough sales to make it profitable. A lot of authors who are self-publishing their backlist probably have lower overhead and can tolerate a slower payback.

Exactly.
If is a rights and investment value decision, not a technical issue.
Traditional publishers also have release schedules that limit the number of releases each imprint brings to market each year so they prioritize fast sellers over long tail releases.

[Catlady]

Originally Posted by fjtorres

I've done technical papers at work and narrative fiction and non-fiction for personal use. A 400 page book can be scanned in under an hour, ocr'ed in 30 minutes, and proofed in 3-4 hours and no special skill is required; it's the kind of work you can give to a summer intern.

In what universe can a proofreader read 100+ pages per hour?

[eschwartz]

Originally Posted by Catlady

In what universe can a proofreader read 100+ pages per hour?

If you just have to check for formatting errors? Probably. Remember, he continued on to say (emphasis mine):

Originally Posted by fjtorres

With the proper tools and workflow the proofing is reduced primarily to looking for broken sentences as the ocr software is effectively perfect when fed 400-dpi grayscale images. For non-fiction, the biggest effort is tagging the footnote/endnote hyperlinks.

Also, I can read about 100 pages an hour on a good day, and both comprehend the words and notice glaring errors.

[HarryT]

Originally Posted by eschwartz

Also, I can read about 100 pages an hour on a good day, and both comprehend the words and notice glaring errors.

But that is not "proof-reading". Proof-reading means comparing the scanned book with the original, comma by comma, word by word, line by line. 10 pages an hour is good going for proper proof-reading, which means 40h work for a 400 page book.

Originally Posted by HarryT

But that is not "proof-reading". Proof-reading means comparing the scanned book with the original, comma by comma, word by word, line by line. 10 pages an hour is good going for proper proof-reading, which means 40h work for a 400 page book.

This is my experience. That is if the goal is to exactly reproduce the original, character by character, and including variation in font face and style.

[fjtorres]

Originally Posted by Catlady

In what universe can a proofreader read 100+ pages per hour?

The one where the scan is clean, aligned, and the software is good.

You're just looking for formatting artifacts to get a clean document, which is what a publisher needs as feedstock for their normal workflow.

It's doable and affordable.
Honest.

Now, securing the rights to do it commercially, that is non-trivial and can be (relatively) pricey if the book is merely a midlister.

[issybird]

Originally Posted by SteveEisenberg

Here's a bestselling no-eBook-available 1994 book with a living, New York City-residing, author who still has (allowing for mergers) the same publisher today:

http://www.amazon.com/Power-Broker-R...s=robert+moses

Author Robert Caro's newer books are eBooks, but his first is not. While there could be some complicated explanation in this case, I think the most likely reason for much I want to read being paper-only is that correcting misscans is expensive.

I've been dying to read this for years, but the pbook is far too unwieldy.

[HarryT]

Originally Posted by fjtorres

The one where the scan is clean, aligned, and the software is good.

You're just looking for formatting artifacts to get a clean document, which is what a publisher needs as feedstock for their normal workflow.

It's doable and affordable.
Honest.

You can get "good" like that, but certainly not "perfect". I've done a heck of a lot of OCR with all sorts of different equipment from hobbyist scanners to pro kit. Even the best OCR packages don't claim an accuracy of more than about 99.99%, which means that, on average, 1 character in 10000 will be wrong - an average of 1 error in 5 pages, which equates to about 80 scanning errors in a 400 page book. Punctuation tends to be far less accurate than text - it's all too easy for a scanner to miss a comma or an apostrophe, or see dirt on the page as a spurious one.

I imagine you did your work with laser-printed documents? These are going to be a lot cleaner - and hence more accurately scanned - than grubby old paper books.

[fjtorres]

Originally Posted by HarryT

I imagine you did your work with laser-printed documents? These are going to be a lot cleaner - and hence more accurately scanned - than grubby old paper books.

The technical reports were vintage 30-40 year old Government Printing Office--likely linotype. The books were mostly hardcovers and a couple of paperbacks. Punctuation was rarely an issue.
The quality of source paper turns out to be a big determinant. Font size, too.

For personal use I processed a vintage ACE double (PEOPLE MINUS X/LEST WE FORGET THEE EARTH), yellowed and falling apart, on my cheapie Canon Scanner (slower but still good enough) and it still did a fine job. The biggest issue was still snipped sentences and paragraphs and those I could easily catch by changing font sizes in word and forcing reflows.

The workflow I used was to save each double page scan as "true page" MS Word and then feed it to Wordpad to get a single column text stream. That went into MS word where I set the paragraph indents to an inch or so and reveal paragraph markers. That exposed most of the non-indented lines that needed to be merged back into the paragraph. The spell checker highlighted any character issues.
(Shrug)

The intended output was a clean readable document, not a print replica. (After all, author manuscripts are rarely perfect when the publisher gets them.) For corporate publishing purposes, that should be *cheaper* than processing a typical author manuscript submission, cost-wise, because it wouldn't require content editing.

The cost of doing a good scan in a corporate environment shouldn't be a deterrent. *If* you know what you're doing and you care about doing a good job. Which, admittedly, not all corporate publishers do, given some of the poor ebooks they have sold in recent times. But that's a different discussion, no?

[Ninjalawyer]

I suspect fjtorres and the article are correct - the major cost is acquiring the rights to the book. From the article:

Over email, Heald explained that he believes that legal framework is the significant reason behind the disparity, not the technological problems of scanning books or those of consumer preferences. "I would expect the gap between music and books to close quickly if copyright law were reformed. Remember that we see a 94 percent ebook rate among public domain bestsellers from 1913 to 1923. Even if books are more expensive to digitize and the market for eBooks is less profitable, we still see high digitization rates when copyright restrictions are eliminated."

Those arguing that this is all a result of the difficulty with OCR are missing the point of the article, and make me think that they haven't even bothered reading it.

TL;DR: Long copyright makes some books hard to find, news at 11.

[fjtorres]

Originally Posted by Ninjalawyer

TL;DR: Long copyright makes some books hard to find, news at 11.

It's really just another pass at the old orphan works issue.
First, you need to find the licensed copyright holder; which is rarely the author, given the length of most tradpub contracts, that are effectively rights sales rather than leases. That can be very expensive, which is why Google and the various goverment orphan work "land grabs" are structured as opt-outs, rather than opt-ins.