Why The Commercial E-Book Market Is Broken

[Steven Lyle Jordan]

Originally Posted by Darqref

Producing backlist books may be harder than it sounds. At Baen, there are lots of books for which it's easy to reprint, but much harder to release electronically. They exist as film, which can be sent to the printers, but must be scanned and OCR'd before electronic copies are available. Baen uses them for filling months with duplicate books, but they have to pay someone to do the scanning.

Okay, given. But it's not like it's rocket science. Scanning and OCR-checking can be given to interns, farmed out to foreign services (who only have to distinguish an incorrect letter from the original text, whether or not they know what it means), or done by summer help.

Such a job needs 3 people: One to scan; One to do the OCR and checking; and one to check behind them. I think most publishers can handle the cost of 3 interns producing from the backlist as many as they can manage.

[jayne80]

Originally Posted by cstross

I'm British. British libraries do not -- as far as I am aware -- have ebook lending programs.

have you checked your library on line?, i'm in the UK and Essex County Council offer ebooks for 21 days for mobi and adobe readers

Tracy

[andym]

Originally Posted by JSWolf

Because Amazon only kept your ebooks available for download for one year. After that, they were no longer available to download. So after one year is up, what you have is what you have. DRM screws you over once again.

I checked with the Amazon site. If you want to buy an 'ebook' they refer you to the mobipocket site.

As far as what they call 'e-docs' are concerned this is what the site says:

Purchasing an eDocument

When you click the button that reads, "Buy now with 1-Click©", your credit card will be charged and your order will be placed. You will then be able to access the eDocument in Your Media Library -- it'll be saved there forever.

[nekokami]

Charlie, that article just keeps coming back!

I've always thought you were spot-on about the "gift-giving" culture. I think the recent Harry Potter release confirms your observations: the competition-- and cooperation-- between the "pirates" to release each volume of HP in ebook form more and more quickly has been amazing to watch.

[Xenophon]

Originally Posted by Steve Jordan

Okay, given. But it's not like it's rocket science. Scanning and OCR-checking can be given to interns, farmed out to foreign services (who only have to distinguish an incorrect letter from the original text, whether or not they know what it means), or done by summer help.

Such a job needs 3 people: One to scan; One to do the OCR and checking; and one to check behind them. I think most publishers can handle the cost of 3 interns producing from the backlist as many as they can manage.

And indeed, they do farm it out. It turns out, however, that getting clean copy is more difficult than it might seem. Arnold Bailey (the web-guy who takes care of these things for Baen) has been through a bunch of different contractors/services/etc. for scanning/OCR/checking. I think he has some reliable choices now, but I seem to recall that it took a bunch of tries to results good enough for publication.

It certainly isn't impossible, but neither is it so cheap and easy as you make it sound. Unless, of course, you don't mind having one or more OCR errors per page.

If you want to learn more than you knew there was to learn about OCR and OCR errors, go hang out on the Distributed Proofreaders fora.

Xenophon

[DaleDe]

Originally Posted by Xenophon

And indeed, they do farm it out. It turns out, however, that getting clean copy is more difficult than it might seem. Arnold Bailey (the web-guy who takes care of these things for Baen) has been through a bunch of different contractors/services/etc. for scanning/OCR/checking. I think he has some reliable choices now, but I seem to recall that it took a bunch of tries to results good enough for publication.

It certainly isn't impossible, but neither is it so cheap and easy as you make it sound. Unless, of course, you don't mind having one or more OCR errors per page.

If you want to learn more than you knew there was to learn about OCR and OCR errors, go hang out on the Distributed Proofreaders fora.

Xenophon

If you look at Gutenberg text you will still find quite a few errors in the text. It is certainly not easy to find them all. Someone (probably more that one person) has to basically read the entire text and has to be alert to the kinds of errors that appear and in many cases you also need to have the original book present and go look it up. There are lots more problems in text that just a simple letter substitution in my experience.

Dale

[Steven Lyle Jordan]

Originally Posted by Xenophon

If you want to learn more than you knew there was to learn about OCR and OCR errors, go hang out on the Distributed Proofreaders fora.

Nah, that's okay... I've done enough of both to already know most of it. And that's why I say it's not that bad.

Over the years, I developed a 2-step process that eliminated most of the errors from scan-and-OCR:
1. Photocopy the pages enlarged to letter-sized pages;
2. Run the enlarged photocopies through high-quality auto-feed printer scanner (like those designed for high-speed network printers like Xerox DocuTech systems) for OCR.

The enlarged pages not only ran better through standardized scan feeders, but the larger letters provided more accurate OCR. It could even speed up the process, in many cases, depending on the equipment you used.

Then you check the copy, and hire a proofreader to check behind you.

In using this method, I've had excellent quality and acuracy out of OCR projects. I repeat: It's not rocket science.

Obviously, you still have to pay your workers, and the better your wages, the better quality of their work. Quality and speed also depend on the speed and quality of the equipment used (and the corresponding cost). But this system works beautifully for most any OCR project.

[nekokami]

I suppose the films could be used to make image PDFs, but most people wouldn't be satisfied with that.

I'd accept quite a few OCR errors in the Baen Free Library, but it would be nice if publishers could recover at least some of their costs by selling backlist ebooks. And I can see that science fiction, in particular, would be tricky to OCR, because of the high incidence of non-dictionary words (including the occasional weird punctuation and even fictional alphabets).

[Steven Lyle Jordan]

Originally Posted by nekokami

I'd accept quite a few OCR errors in the Baen Free Library, but it would be nice if publishers could recover at least some of their costs by selling backlist ebooks. And I can see that science fiction, in particular, would be tricky to OCR, because of the high incidence of non-dictionary words (including the occasional weird punctuation and even fictional alphabets).

SF and fantasy can be a bit trickier... but that's why you put 2 people on the back-checking job, and they'll have to rely on actual reading, not Word spell-check.

In fact, many people do proofreading of even trickier text every day, in technical industry and government documents that use proprietary words, industry-specific acronyms, equations or symbols that a computer-based spell-checker can't check, or in some cases even flag. The best way to do it is to hire people that can give full attention to the subject, especially if it is an interest of theirs, but preferably if they have indicated enough accuracy with that type of genre to be considered reliable with it. It's not impossible, it just requires dedication to the effort.

A lot of people wonder about the future of editors when the world starts publishing digitally outside of publishing houses... well, this might be one of the areas that editors will end up taking over, since they should have the training and experience to do back-checking.

And the way I see it, besides editors, with so many people out there looking for work, and all the people that are house-bound, you have a sizeable population that could do the text-checking job for just a reasonable salary. If editors don't go into back-checking, they would end up managers of the back-checkers.

[slayda]

Originally Posted by DaleDe

If you look at Gutenberg text you will still find quite a few errors in the text. It is certainly not easy to find them all. Someone (probably more that one person) has to basically read the entire text and has to be alert to the kinds of errors that appear and in many cases you also need to have the original book present and go look it up. There are lots more problems in text that just a simple letter substitution in my experience.

Dale

Actually they use volunteers and have multiple levels of proofreading and formating. You don't get the entire book. You only get one page. When you're done, it is turned in for the next level. Then you can get another page in that text or some other. It can be both enlightening and frustrating at the same time, trying to guess what you missed.

[DaleDe]

Originally Posted by slayda

Actually they use volunteers and have multiple levels of proofreading and formating. You don't get the entire book. You only get one page. When you're done, it is turned in for the next level. Then you can get another page in that text or some other. It can be both enlightening and frustrating at the same time, trying to guess what you missed.

I understand volunteers but generally they are pretty dedicated but the results still have a lot of errors. It is frustrating for everyone, including the guy trying to read it later.

Dale

[slayda]

Originally Posted by DaleDe

I understand volunteers but generally they are pretty dedicated but the results still have a lot of errors. It is frustrating for everyone, including the guy trying to read it later.

Dale

I agree. That's why when I did it, I tried to be very careful. I actually prefer to proofread on my Palm T/X. I highlight & bookmark each error I find & later correct them on my PC. Of course this is my own scanned/OCRed books.

[nekokami]

Steve, I suspect managing the back-checking would turn out to be as expensive as paying the checkers themselves. Any way you look at it, its an expense a publishing house has to consider when deciding whether (or how quickly) to publish their backlist as ebooks. It's not free, and there are a lot of backlist books.

I still hope it will happen, of course! But I acknowledge that there's an expense involved.

[Steven Lyle Jordan]

Yes, they're not going to do it for free. But I can't help but think that this is something that can largely be contracted out to shut-ins, domestic and/or foreign companies and co-ops, people who can use the work and (for whatever reason) have few options. This has the benefit of bringing costs down, while simultaneously giving work to those who could use it. And since it's not a high-pressure task (since it's not time-critical, mainly), checkers can work at their own pace.

I know, the idea of contracting out such work to a foreign group might seem strange, but with the proper monitoring and contract reviews, such a system is eminently possible. And anyway, if you don't want to farm it out to foreigners, there are plenty of shut-ins in everyone's native country who can take the work (many of whom are doing menial work now, and still struggling to compete with cheaper markets).

One thing this world can really use is more opportunities for people that are geographically or physically isolated from the markets, so that they can earn a decent wage and live a better life. There are places/people in the world that would kill for opportunities like that, to allow them to earn money and take care of themselves, and it's a proven way to improve the quality of life in disadvantaged groups/areas. And I think the task of OCR/checking would be perfectly suited to such programs.

[nekokami]

I think the Mechanical Turk project at Amazon is showing that there are people willing to do this kind of work-- but so far, that's not earning much money for the workers. http://www.salon.com/tech/feature/20...rks/index.html