Some perspective on why ebooks are so filled with errors.

[Hitch]

There's enough vitriol here to run some cars for millenia.

Look, there are a variety of processes to get text into ebook form. There seems to be a confusion of the methods in this thread.

1. A new book is being produced; the author has provided a Word (or other) file, and the book is being created in both print and ebook form by a publisher or producer;
2. A new book is being produced, in print only; and some time after that, the publisher/author decides to create an ebook therefrom;
3. A new book is being brought to life as an ebook only--and the source material is Word or PDF; and/or
4. A book exists in backlist, in print only, and the book has to be OCR'd to be brought to digital life, before being made into an ebook.
5. There are other scenarios, but these are the main; no point in digressing into painstaking minutiae, here.

In theory, when you look at #1, the number of proofing errors should be identical, from print to ebook. It rarely works this way, because as elcreative pointed out, I believe, the print layout process creates a massive file unsuited to e-production. Most print layout companies use InDesign or something similar (which outputs seriously crap ePUB files, don't let them kid you)...and the laid-out book is what is sent out, in galleys, to the author/editors for proofing, not the source material. This is where the process diverges, and you end up with two differing sources; the marked-up INDD file, and the original source material. Generally, what happens is that the INDD file gets polished, sent to the printers...and the Word file (or OO, or whatever) gets sent to someone like Aptara, to be spat out as the ebook. So: that's process/item #1.

#2: See #1. I get a ton of Word files that have to be edited to match the original print book; or I get the PDF of the final print book, and we have to have the author proof it for any conversion errors, which of course, do happen, although fewer nowadays than, say, 2-3 years ago;

#3: that's on whomever is the publisher. I'd bore you all to tears with the stories of the crap we get; but suffice it to say that no matter that I make our clients SWEAR that they're giving us the "final" proof--it is never so. I cannot remember the last time we had a book go through without copyedits. Literally. And we add 1 client/day, and most of our clients are multi-book authors; in general, we're putting out +/-1500 books/year.

#4: And here is where the OCR scenario, replete with Abbyy Fine Reader, usually, comes into play. This has ZIP bearing on 1-3. At this point, yes, you have to pay someone to proof for OCR errors. In general, just for scanning errors, at a pro scanning firm, it's $1/page (just for the proofing). You'll still get code errors (like random paragraph breaks that are invisible until the thing is exploded, soft-hyphens running rampant, etc.), but a good conversion house {ahem} will regex (search and replace) those things. But still...now the author or publisher is into it for roughly $1.50 a page, without an actual line editor-style proof; the scan proof really is searching for ligature errors, and that type of thing. From a competent scanning house, and I cannot emphasize that word enough, that knows what is needed to push print to eventual ebook, you can get a file that a good convo house can make into a decent ebook with nominal errors. Ergo, a 300-page typical fiction title starts its conversion to ebook--before it ever gets to someone like me--at $450.00. That's BEFORE it gets to a convo house; and this is for an author (or indy publisher, or backlist pubber, whatever) that's going to sell for $0.99, more than likely; maybe $2.99).

So, if we're all going to sling mud here, let's make sure we squean about the right cranial smoke. If we're gonna talk big publishers--generally, they're either doing 1 or they're sending an unread Word file to the Aptaras of the world--no edits, no proofing. If we're talking scans, let's talk scans, and let's not mix it all up. Reamde certainly wasn't made from a scan or OCR. It's apples and oranges.

BTW: the "beta reader" idea--I have authors that would kill for reliable beta readers. Beta readers are, Keryl, one of those generally-accepted, vaguely conceptualized ideas of "things that exist" that, quite bluntly, don't. Sure, if you're Dan Brown or Laurell K. Hamilton or one of those people with troos, it's easy to find beta readers; but for the average mid-lister (and that's who makes up the vast majority of publishing, folks), it's nearly impossible. Again, that's like the infamous pretty teenage girl who will come over and exercise and take care of your horse just for the love of it--they don't exist, either. And, btw: most of the beta readers are fine for plotline development, but they're not proofers. Not the same thing at all. Like test screenings--they're only looking for audience reaction, not line-by-line "edits." Beta readers tell the authors (like Charlaine Harris) that they like this plotline, don't like this development, etc.--but they don't proof.

(And farm out proofing to India? No. Trust me on this.) And lastly--for those of you that think that proofing is easy, I'd recommend you try it. Particularly, try it with a book you hate. Or a writer who sucks--not somebody whose book you can't wait to put your hands on.

HTH, from the sausage factory,
Hitch

[RichL]

I'll toss in a couple of my own observations from having assisted in assembling a book of poetry. It was a self published short run and the printer gave the gal a good price on it if she submitted the disk ready to run.

The book was assembled with Adobe something (Acrobat? Publisher?). If a letter was replaced or we did a cut and paste from one section to another what happened in the program is that it would stay in the original form but a code would be inserted that, in essence, said 'Instead of printing this here print it five lines down.'. What it really boiled down to is that the program was designed to give instructions to printing presses or some such, not computer screens.

If you called that program up today to make an ebook from it you would go buggy before you got half way through trying to correct the spelling and formatting. I realize that not being in any sense a professional in the publishing field the above is not real clear but I'm sure that Hitch can clarify if necessary.

The second thing is that I have done proof reading of original OCR scans and I would be happy to do it again. Starting at 25 bucks an hour and up depending on how good the scan is. It is one of the most time consuming, mind numbing, eye straining jobs I ever undertaken. As a matter of fact I told my boss of the time 'First time, last time, there ain't no discussin it further.' :-)

[JeremyR]

There is a place what will scan a book for $1 per 100 pages, and then OCR it for another $1.

1 Dollar Scan it's called. So no, it doesn't start at $450. Oh I'm sure some firms will charge that, but they are making an obscene amount of profit.

As to correcting the OCR, sure, it's not fun. I've done it on some books I grabbed from the Internet Archive. But it's hardly the worst job in the world. Try doing manual labor sometime - digging ditches, working on a farm, etc.

I have, and I tell you, I'd much rather be proofreading books for minimum wage (except for course, companies apparently can't even be bothered to try to hire people to do it...). Even though I'm sure most publishers have no qualms about giving out huge bonuses to celebrities and politicians for book deals (most of which bomb) or cocktail parties and such.

[elcreative]

And thanks to carpetmojo, vxf, Hitch and RichL... voices of reason and simple understanding of reality rather than wishful thinking... and on that note, before everything repeats in ever decreasing circles, I can only say, "Farewell sweet thread, and possibly farewell most if not all of MR as the sensible voices of reason and open discussion sink slowly into the sea of idiocy and vitriol..."

[xg4bx]

i just reported Norman Spinrad's The Iron Dream. the book is littered with typos, poor formatting, bizarre symbols replacing letters and spaces and they charge a full $8 for the book. no publisher name is given either, its just posted by whoever.

the only other option for a reader is to buy a hardcopy which ranges from $58-$300. it sucks that the only current version "in print" is a disgusting and angering mess.

[howyoudoin]

Originally Posted by JeremyR

There is a place what will scan a book for $1 per 100 pages, and then OCR it for another $1.

1 Dollar Scan it's called. So no, it doesn't start at $450. Oh I'm sure some firms will charge that, but they are making an obscene amount of profit.

As to correcting the OCR, sure, it's not fun. I've done it on some books I grabbed from the Internet Archive. But it's hardly the worst job in the world. Try doing manual labor sometime - digging ditches, working on a farm, etc.

I have, and I tell you, I'd much rather be proofreading books for minimum wage (except for course, companies apparently can't even be bothered to try to hire people to do it...). Even though I'm sure most publishers have no qualms about giving out huge bonuses to celebrities and politicians for book deals (most of which bomb) or cocktail parties and such.

Bingo!

This man knows to get to the point.

[Cyberman tM]

Originally Posted by Hitch

Most print layout companies use InDesign or something similar (which outputs seriously crap ePUB files, don't let them kid you)...and the laid-out book is what is sent out, in galleys, to the author/editors for proofing, not the source material. This is where the process diverges, and you end up with two differing sources; the marked-up INDD file, and the original source material. Generally, what happens is that the INDD file gets polished, sent to the printers...and the Word file (or OO, or whatever) gets sent to someone like Aptara, to be spat out as the ebook. So: that's process/item #1.

Could you expand on that?
I have no doubt it's true, but it sounds ridiculous, so say the least. (The part with two separate sources, not the part with ePub - that I certainly believe, but it's only a intermediate problem, I wouldn't believe it's an unsolvable problem).

Why is there not a single file format that is used for corrections/layout/print/whatever?
Why keep two (three?) "sources" running concurrently?

[mr ploppy]

Originally Posted by Keryl Raist

You know, for popular authors this would be insanely easy to do. Video games have lists of people in line to be beta players. How hard would it be to get people to do that for books? Sure you'd have some spoilers creeping out, but you'd also get cleaner books for minimal costs.

Movie studios do this. They don't pay test markets to go see movies and let them know what needed work. Why not have test readers?

That's basically what they do, they're called advance reader copies, and they replaced professional proof readers years ago. Two problems with that, proof reading is a rare talent, and any changes only get made to the book version.

[mr ploppy]

Originally Posted by Hitch

In theory, when you look at #1, the number of proofing errors should be identical, from print to ebook. It rarely works this way, because as elcreative pointed out, I believe, the print layout process creates a massive file unsuited to e-production. Most print layout companies use InDesign or something similar (which outputs seriously crap ePUB files, don't let them kid you)...and the laid-out book is what is sent out, in galleys, to the author/editors for proofing, not the source material. This is where the process diverges, and you end up with two differing sources; the marked-up INDD file, and the original source material. Generally, what happens is that the INDD file gets polished, sent to the printers...and the Word file (or OO, or whatever) gets sent to someone like Aptara, to be spat out as the ebook. So: that's process/item #1.

Like I said, bad management. When you get the list of mistakes from the beta-readers that should be sent to the writer to sort out. Then when that's done (and someone else has checked they've all been fixed) the original word processor file goes back to the print designer, and a copy goes to the ebook designer.

[RDaneel54]

Originally Posted by Hitch

Look, there are a variety of processes to get text into ebook form. There seems to be a confusion of the methods in this thread.
HTH, from the sausage factory,
Hitch

Hitch, thanks for the detailed explanation.

My little bit of experience was dealing with publishing in the U.S. DoD. I wrote the materials and we had in-house editing and proofreading. The books were then contracted to the printers. Everything was about layout for the printers.

No matter how much I double checked the work of everyone after it left my hands, I almost always had to deal with a certain amount of errors in the end-product. Since I was also teaching from the books, it was embarassing to have to explain some of the errors to our customers/students.

Over the 12 years I dealt with this process, I became more and more insistent on as much review as I could possibly get away with. Our editor realized I was not going to accept less than 100% effort from everyone. It got better, not perfect, and we had setbacks, but overall our endgame was substantially improved.

One process we had to adapt our materials to was on-line presentation. A nightmare from the beginning. When I left, I was still fighting the battle to convince our publication staff that this was a different medium and needed different process from beginning to end. It was very difficult to change the mindset. I was unable to convince them that editing and proofreading needed to be done on the web presentation, not paper copies.

[EowynCarter]

Yes, prof-reading is a long and tedious job. But I'm paying the book...

I've rarely been bothered with errors with p-books, but e-books, really...
One of was just un-readable, the kind of things that could hardly escape anyone having open the book and just read the fist page..

I've seen books with enough typos to be bothered by it. And being dyslexic and English not being my native language, lots of them escape me...

[Catlady]

It's amazingly obvious in this thread who the people are with real-world publishing experience.

As was already stated, proofing errors, typos, OCR errors, and formatting errors all seem to be jumbled together in this discussion, even though they represent different problems. But what I am really wondering is just what books are filled with all these horrendous errors, because I simply haven't seen them.

Almost all my book purchases are from the big Agency publishers, often with price tags of $12.99. Other than a few run-of-the-mill typos that creep into every book, the main quirk I've noticed is the removal of hyphens from all hyphenated words (e.g., custom-built becomes custombuilt), which is presumably a computer error. I've also seen a few instances where display type (e.g., a dropped cap in the chapter opening) is not displayed correctly, or margins for an extract are too wide--again, small stuff.

Since in some discussions there seem to be a lot of people who loudly proclaim that they will not go over a certain price point, or will not buy from the Agency publishers, or will not buy a DRM book, I'm wondering if either (1) you get what you pay for, or (2) there really aren't that many errors after all.

[HomeInMyShoes]

^Life and Death are Wearing me Out by Mo Yan had some consistent errors in it that were probably the result of OCR or bad global fixes. This was Arcade Publishing. While not a gigantic publishing house, they have some pretty big name authors. Currently, this book lists for $16.47. I picked it up for $3 at an Amazon sale. Currently, that's a steep price if it's still got those errors.

[Keryl Raist]

Originally Posted by mr ploppy

That's basically what they do, they're called advance reader copies, and they replaced professional proof readers years ago. Two problems with that, proof reading is a rare talent, and any changes only get made to the book version.

Hmmm... I've gotten ARCs and have never been asked to make any changes to them. (And yes, they often needed them.)

And yes, proofing does take skill. I'm not suggesting that any given Beta returns a pristine copy. But get ten or so of them and you've likely got a much better base document than you had before. After all, I don't think any of us are expecting perfection, just significantly better.

As for only using those changes on the pbook, well that's not an insurmountable issue.

After all, producing a print copy of a book is not rocket science. For an ARC it doesn't have to be high art, it just has to be legible. Then once you get the corrections back and implemented, split the doc into it's final parts.

If I, self-published author, can figure out that splitting my document into it's different formats is the last step in the process, you'd think the big boys could realize this, too.

[Hitch]

Originally Posted by mr ploppy

Like I said, bad management. When you get the list of mistakes from the beta-readers that should be sent to the writer to sort out. Then when that's done (and someone else has checked they've all been fixed) the original word processor file goes back to the print designer, and a copy goes to the ebook designer.

@mr ploppy: {edited...I started out with a rant about clients. I'm human, but I'm over it now}....

When we do POD as part of our process here, that is precisely what we do; we do the POD version first; we send it in PDF to the author (we no let authors touch their books once we've had them in our hands--we have had some mighty bad experiences with that experiment), we get back a proof sheet, we make the edits, author signs off, and THEN and only then do we export the html and make the ebooks. It's a pain, but that's the only way to be sure we conform the copies.

@JeremyR: Yes, they'll OCR it for $1.00. That's not proofed. What I said was, to have a professional scanning company scan and proof it was $1.50/page, because $1.00 page of that is proofing. You can have BlueLeaf do a scan job for el cheapo, no problem. Trust me when I tell you that you either pay for the proofing, or you end up paying someone like me to make the nine bajillion corrective edits that didn't get made when you didn't proof the bloody thing in the first place. I wish I had a dollar for every author that called in hysterics over an el-cheapo scan & ocr job. (My use of "you" is generic, in the previous paragraph). If you, @JeremyR, want to proofread for minimum wage, please revert to me privately; I have any number of clients that would love to use your services. (Also: if you can reliably copyedit epubs, revert as well). Of course, see the other side of the coin, next, with @RichL...

@RichL--QED. O'course. Wish I had an author that would pay that. My average line editor right now (super-proofer, IOW) is making $10-$12/hr.

@Cyberman tM: Yes, of course it sounds ridiculous, but the bottom line (no pun intended) is that print layout ain't just slap it in Word and bingo-boingo, a book pops out. There's a lot that goes into it. DT Publishers have had a process for dog's years, in which they send the book for layout, it gets put into arc's (Advance Review Copies) laid-out, and everyone proofs for everything. Typos, grammar, hyphenation, widows, orphans, you-name-it. The ARC's can be proofed manually (people use stickies, or write on them); tags; or can be annotated in Adobe. But note: what's being edited is a PDF, really. The source is not being edited. An output (generally) from InDD, or in rare instances nowadays, Quark. Now, because INDD's epub output is not great, one of two things happens: either a crappily-output epub from INDD is sent to the distribution point, OR the original Word version (or a text-only-output version) is sent to the conversion house, if the print layout person(s) don't "do" ebooks. If the former, the formatting may suck; if the latter, it does not have the same edits that were made to the ARC's. This is one of the reasons that INDD is trying very hard (gentle cough) to make an "insta-epub" button, altho they are still a long way from that. And, as near as I can tell, nobody at Random House, Harper, yadda, have read Liz Castro's very fine book on ePUB's, specifically and primarily from INDD. Does that clarify?

@RDaneel54: I don't disagree with you. My position, however, is that I'm basically a modern-day printer. We're ebook producers-not publishers. As I tell my clients, (about which they b***h), we're not the publishers, it's not our job to proof the books--it's theirs. If I had to proof every book we do, word-by-word, I'd have to charge a hell of a lot more. We try very hard to output quality books--but it's a little outside my purview to start editing the "badness" I see, and I see a lot of it. When I pop open a book for quotation, and the first thing I see is that it has a "forward,' I know I'm in the deep sheepdip. Is it my job to change that? Or warn the author? Or...? When the average fiction (or plain narrative text) title is made (in ebook formats), done and dusted, epub and mobi, for less than $200, and most for under $175? Can't be done. Oh, I'm sure someone (who is either doing books for love, or doesn't have to lpay employees) will pop back on here and tell me I'm full of crapola, but it's hard to make a quality book in anything less than 3 hours, and that's from super-clean source--which we never get. Usually it's closer to 6 hours, and of course, let's not overlook all the cleanup we do because authors send us covers in PDF; or the publishers send the wrong book (happens at least once a week), or the source images are horrifically bad, or...the list, kids, is pretty endless.

Did I manage to respond to everyone? If I missed, my apologies...but I'm "away" from the office a bit too much answering this thread as it is...

Best, HTH,
Hitch