Some perspective on why ebooks are so filled with errors.

[Keryl Raist]

Originally Posted by Hitch

BTW: the "beta reader" idea--I have authors that would kill for reliable beta readers. Beta readers are, Keryl, one of those generally-accepted, vaguely conceptualized ideas of "things that exist" that, quite bluntly, don't. Sure, if you're Dan Brown or Laurell K. Hamilton or one of those people with troos, it's easy to find beta readers; but for the average mid-lister (and that's who makes up the vast majority of publishing, folks), it's nearly impossible. Again, that's like the infamous pretty teenage girl who will come over and exercise and take care of your horse just for the love of it--they don't exist, either. And, btw: most of the beta readers are fine for plotline development, but they're not proofers. Not the same thing at all. Like test screenings--they're only looking for audience reaction, not line-by-line "edits." Beta readers tell the authors (like Charlaine Harris) that they like this plotline, don't like this development, etc.--but they don't proof.

As I believe I said, it would be easier for the popular authors.

However, back in the day (this being the late 90's) I lived in and among the English Majors/Writing MFA candidates, and they were all dying for some sort of in with publishing houses. I'd think being able to put proof reader/beta reader on the CV would be a start in the right direction, but I could be wrong. Maybe, with this job market they've got so many better opportunities they just can't be fussed to build a CV. Maybe they no longer care about being more competitive than the next English Lit MA. Or maybe Random House is too damn stuck in it's ways to put an ad on Craig's List looking for Beta Readers.

I'm as low down on the publishing list as you can get. I'm a one person operation with a less than shoestring budget and I can find beta readers. Are they all great? No. Do they combined provide me with a cleaner copy than I could get on my own? Yes.

Midlisters are midlisters because they've got fans. Maybe not tons of them, but a decent enough number to make churning out more books worthwhile. Get ten of them to give the ebook version a once over to catch the nasty bits, and your final product might not be perfect, but it will be better.

Since we're talking ebooks, it's not like this is prohibitively expensive. Each additional copy of the base document costs them nothing. Advertising for betas doesn't have to cost much either. (For midlisters a note on the facebook page/blog/website should suffice.) Putting the final corrections into the proof can be farmed off onto the author. It will make the books take longer to put out. How much adding an additional month or two to the publication schedule costs, I don't know. I do know that many authors get their book finished months before it hits shelves.

I know this technique works on the micro scale. I'm not seeing a compelling reason for why it can't be ramped up to the macro scale.

[mr ploppy]

Originally Posted by Keryl Raist

If I, self-published author, can figure out that splitting my document into it's different formats is the last step in the process, you'd think the big boys could realize this, too.

They probably do realise it, but all their processes are geared towards producing real books so it will take a major change in thinking for them to do it a more logical way.

Don't forget that we are just one-man production teams handling everything, so it's very easy to just make the changes to the word processor file. Once you involve other people, whether they are editors, layout artists or whatever, you end up with more than one copy of the word processor file floating around and very little control over what happens to it.

That's why it's a problem of management more than anything else. Only the writer cares enough about the product to get it right. Everyone else is just interested in the money.

[toddos]

Originally Posted by mr ploppy

Don't forget that we are just one-man production teams handling everything, so it's very easy to just make the changes to the word processor file. Once you involve other people, whether they are editors, layout artists or whatever, you end up with more than one copy of the word processor file floating around and very little control over what happens to it.

I'm neither an author nor do I work in the publishing industry. However I do work in the software industry, and we've had that problem figured out for decades. When you have more than one person working on source, you must use some form of version control (and it's not a bad idea to use source control even if you're a one-man show, just because it's always useful to be able to branch off, experiment with changes, and still know that you can get back to the original if necessary). The more robust, the better, so that you can easily go back in time to back out bad changes, track who made what changes, see diffs, etc. It's a solvable problem with an entire industry proving that it works. There's no reason it can't or shouldn't be used for publishing.

[Keryl Raist]

Originally Posted by toddos

I'm neither an author nor do I work in the publishing industry. However I do work in the software industry, and we've had that problem figured out for decades. When you have more than one person working on source, you must use some form of version control (and it's not a bad idea to use source control even if you're a one-man show, just because it's always useful to be able to branch off, experiment with changes, and still know that you can get back to the original if necessary). The more robust, the better, so that you can easily go back in time to back out bad changes, track who made what changes, see diffs, etc. It's a solvable problem with an entire industry proving that it works. There's no reason it can't or shouldn't be used for publishing.

I do this with my books. Word has both a change tracking feature which allows each user to put comments/changes in a different color, and then merge or unmerge as you so desire.

The base document sits all by it's lonesome, doing basically nothing for most of the time. The versions go out and play. Eventually they get merged together, with different changes as needed. Then the new, merged doc gets compared to the original, and then yet another new doc of the original and merged version gets made. That doc is then the one that gets to become the eventual book.

But at each step copies are kept. I might have seventeen versions of my novel before I get to the one I'm going to print from.

[Keryl Raist]

Originally Posted by mr ploppy

They probably do realise it, but all their processes are geared towards producing real books so it will take a major change in thinking for them to do it a more logical way.

Don't forget that we are just one-man production teams handling everything, so it's very easy to just make the changes to the word processor file. Once you involve other people, whether they are editors, layout artists or whatever, you end up with more than one copy of the word processor file floating around and very little control over what happens to it.

That's why it's a problem of management more than anything else. Only the writer cares enough about the product to get it right. Everyone else is just interested in the money.

I have a feeling that, in the end, we'll end up with a combination micro-macro publishing system.

It is easier for authors to keep the quality control up when we do it ourselves.

It's easier for big publishers to handle marketing and logistics.

A system where each of us does what we do best would probably work out better for all involved, including the readers.

[toddos]

Originally Posted by Keryl Raist

I do this with my books. Word has both a change tracking feature which allows each user to put comments/changes in a different color, and then merge or unmerge as you so desire.

The base document sits all by it's lonesome, doing basically nothing for most of the time. The versions go out and play. Eventually they get merged together, with different changes as needed. Then the new, merged doc gets compared to the original, and then yet another new doc of the original and merged version gets made. That doc is then the one that gets to become the eventual book.

But at each step copies are kept. I might have seventeen versions of my novel before I get to the one I'm going to print from.

I've never had good luck using Word's change tracking features (for writing software specs, not manuscripts ) and would much rather rely on CVS, Subversion, Git, etc to manage versions. Of course those work best with plain text documents, which Word docs are not, but HTML docs are plain text ...

(and Word's change tracking plus Sharepoint's built-in version control for Word documents prove that Word can be handled by a version control system, you just need something that understands the format rather than something purpose-built for plain text source code. I suspect there's a niche market out there for this type of software, if someone were interested ...)

[Keryl Raist]

Originally Posted by toddos

I've never had good luck using Word's change tracking features (for writing software specs, not manuscripts ) and would much rather rely on CVS, Subversion, Git, etc to manage versions. Of course those work best with plain text documents, which Word docs are not, but HTML docs are plain text ...

(and Word's change tracking plus Sharepoint's built-in version control for Word documents prove that Word can be handled by a version control system, you just need something that understands the format rather than something purpose-built for plain text source code. I suspect there's a niche market out there for this type of software, if someone were interested ...)

I'd imagine there's a market out there for just about anything.

I'm sure if I was writing software, I'd have something other than Word in my toolbox. Alas, I'm not a coder. I know as little HTML as I can get away with, and that's pretty much it.

My current version of Word has a versioning tool. I haven't really used it much (I've had Office 2010 for less than two weeks now.), but it looks promising.

[JSWolf]

Originally Posted by jupstin

Are you saying that publishers would hire more editors to produce acceptable ebooks if only there were qualified people available for the job?

It's not more editors that are needed. It's people who can look at the process, see what's the matter and get it put right. If that means the publisher needs more people to be able to edit the Word document so that will be the up-to-date source instead of the PDF, then so be it. If it means more people to read after the OCR then so be it. If that means actually viewing the eBooks to check formatting, so be it.

They need to do it right and most of what they do now is wrong. It doesn't always take much time to fix shoddy formatting. So that should be part of the process. Reading OCRed eBooks should also be part of the process. PDF should not be involved at all.

[JSWolf]

Originally Posted by murraypaul

Were the words originally in italic or bold?
I've seen OCR conversion issues when spaces are lost when those tags are stripped out.

Nope. Plain old non-italic, non-bold uses of the words people and couple. They just lost the space in front in a number of cases.

[Elfwreck]

Originally Posted by Hitch

BTW: the "beta reader" idea--I have authors that would kill for reliable beta readers. Beta readers are, Keryl, one of those generally-accepted, vaguely conceptualized ideas of "things that exist" that, quite bluntly, don't.

When the authors come out of a fanfic background, they generally have (and can find new) beta readers.

And, btw: most of the beta readers are fine for plotline development, but they're not proofers.

Depends on the beta. I know how to contact plot betas, SPAG betas, Britpick betas, historical-detail betas, and various other "read your book & check for errors" people.

(And farm out proofing to India? No. Trust me on this.) And lastly--for those of you that think that proofing is easy, I'd recommend you try it. Particularly, try it with a book you hate. Or a writer who sucks--not somebody whose book you can't wait to put your hands on.

I've done proofing for books I hate. While detailed, careful proofing is tedious (and therefore expensive), a basic "read the first 10 pages and use a search-and-replace to fix the errors you can tell are going to run rampant through this book" would improve a *lot* of commercial ebooks created from scans.

Books have been released with gross errors on the title page. This is not "we can't afford detailed proofing for a small sideline part of the business" (and nevermind that ebooks are rapidly becoming the main part of some businesses); it's "can't be arsed to assign a single human being the job of glancing at the finished product before we ship it out."

*That* they could afford. Assigning someone to open the book, confirm that the title/author/publisher info is accurate (wouldn't want a typo in the publisher's name, right?), that the TOC does what they want it to, and that the reader won't get five pages into it and say, ugh, can't read this; never buying another book like this again.

I'm not seeing anyone argue that publishers REALLY MUST provide the line-by-line proof for ebooks that they do for pbooks (although, given that they charge about the same price, that does seem reasonable), but that they should flip through the book, one readable screen at a time, to see if any awful errors get in the way of reading the story.

Most of the problems come from publishers' assumptions that they don't need to adjust their workflow at all to develop a new line of products, which is baffling.

Layout-format ready-for-print files are indeed cluttered with markup and other details--but a system to take that file & convert to HTML or XML suitable for ebook import would be a lot less work than running the ebook version through another round of proofreaders (and getting a *different* batch of errors caught.) And one decent programmer should be able to write a script/program that converts all the standard markup language to the simplified HTML used for most commercial ebook production (leading and kerning are generally dropped; chapters don't need to start 1/4 of the way down the page, and so on); that's a one-time expense, with a small update every time the software changes, rather than an ongoing cost for every book.

But they don't want that; they want to take the final PDF and throw it into Calibre and call the export-to-mobi & epub their official ebook versions. Or they want to take FineReader's auto-OCR results and use them.

And they want to charge slightly more to half-again what paperbacks cost for that product. THAT'S the real sticking point; if they charged $3 for their ebooks while selling paper for $8, there'd be a lot less complaints. Since they want the same profit from ebooks as pbooks, we figure they can put the same level of effort into it.

[JSWolf]

Originally Posted by Hitch

But on a super-large scale, like one of the big houses (say Random House), I can't imagine how they'd proof all those titles. I really don't. It's easy to say "oh, they're lazy," but putting out really good ebooks isn't significantly easier than putting out a good print title; and proofing one doesn't replace proofing the other--and as someone else pointed out, a good proofing can run $750-$3500 or more.

For a new book, what are they using as the source? If they have a good document to work from that's not PDF, and that document is updated for any changes since the book was submitted electronically then there should be little room for these sorts of WTF errors. If the process is streamlined so it works with say a Word document, then there should be little need to have to read the eBook other then to check formatting. But what they do boggles the rational mind. I can take a Word document and convert it to ePub without worrying about inserting errors not already in the document. The only thing that would need to be done is to fix the formatting and the internal code.

How do we get things like missing spaces, missing letters, missing word, run on formatting, sections out of place, and other oddities in new eBooks that should not have to be run through the OCR process?

[JSWolf]

Originally Posted by Hitch

Y'know, Kevin8or: honestly, there aren't. I know everyone thinks that there are affordable college grads up for grabs on every corner--but not at any type of piece-rate that is remotely affordable. I'm not being pissy about this, but as someone who does hiring moderately often, and just went through a spate of bodies for an admin ass't., it ain't as easy and affordable as it sounds. Most of the grads I interviewed wanted more money per hour (essentially) than my production crew leaders get; the latter of which are line, so earning me money, not staff, which don't. Nor will I make you roll your eyes by regaling you with my not one, not two, but myriad debacles with hiring new proofers and epub-editors, vis-a-vis your earlier comments about "outsourc[ing] the file to a low-wage proof-reader or two" and sending it back to "someone" in-house who makes the changes. That person in-house? Has to be paid. There's no magic wand that makes those changes. We find we can make about 30, maybe 45 edits to an epub in an hour, tops. We get some epubs with over 200 copyedits. I have to pay "someone" for those hours. That ain't free, and as I already said, the "low-wage proof-reader" is fantasy right up there with the belief that when you stealth-fart, no one smells anything, and no one knows it's you.

So how much would a low-wage ePub editor make doing this?

[Fbone]

Originally Posted by xg4bx

i just reported Norman Spinrad's The Iron Dream. the book is littered with typos, poor formatting, bizarre symbols replacing letters and spaces and they charge a full $8 for the book. no publisher name is given either, its just posted by whoever.

the only other option for a reader is to buy a hardcopy which ranges from $58-$300. it sucks that the only current version "in print" is a disgusting and angering mess.

B&N says it's published by Norman Spinrad himself.

[ficbot]

I don't see why they can't revive the galley proof and send it back to the author for a final check before they 'ship' it.

Saying 'it costs money to fix mistakes' is btw NO EXCUSE AT ALL for shipping out a shoddy product, because you know what? It costs money to buy stuff too. And why should *I* be the one to shell out the money instead of you when you shelling it out means an acceptable product and me shelling it out means you've just scammed me and every other customer who trusted you to meet a minimum standard?

[Hitch]

Originally Posted by JSWolf

So how much would a low-wage ePub editor make doing this?

@JSWolf: Let me put it to you this way, without getting into the specifics of my business--which aren't the business of anyone here--whatever you think it might be, it patently isn't enough (and trust me: it's more than you think). I've fired more epub "editors" than I like to think about. Part of it, amusingly enough, is that the html editors get pissed off at the authors, who tend to be pissy in their proof sheets. It would be funny if it didn't cause me so much agita. I can't tell you how many ePUBs I've had to edit myself because I daren't let the Crews see the nasty comments from the authors--almost always about errors that they (the authors) themselves made, and blame on my Crews, or because they don't know a hard return from a ball return.

If I may--the reason that some of you are struggling with "how easy" it would be to single-source this (go from final "print" source to epub to mobi, etc.) is because, quite bluntly, you don't do this for a living, and thus, don't know what really happens when print is involved. This is not a criticism; it's a statement. In the print world (and we do POD PDF's), what really happens is, many of the things that get changed are only visible, and knowable, and hence fixable, when the book is put into print format. For example, if you are almost any author being legacy-published, and the print formatter ends up with a chapter with an orphaned 5 words, dangling precipitously onto an horrifyingly blank page, there's no magic stick that fixes that. At RH, they pick up the phone, call the author, and tell him/her to lose 5 words--or have the "real" editor (not typing editor) do it. Have a sentence that, heavens forfend, has a normally-hyphenated word that then hyphenates syallabically in the second word? You have the author or editor add a word or two...or take one or two out. Hyphenation takes a looong time to do--if you're going to do it correctly. Doing it crappily takes minutes. And making it look decent means a modicum of rewriting--by somebody.

Hence...the print book is now different than the source; and we're back at square one. We convert PDF's, from printers, all the time; and those make for very good ebooks. We convert Office and OO all the time, too; and Wordperfect, although less nowadays; these only make as good an ebook as the effort that was put in, in the first place; GIGO, as we all know too well. This is precisely why--although it was counterintuitive to us, initially, ('cuz we be nerds, not publishers) to do it this way--we do POD first, THEN we export the final "source" and make the ebooks--because we learned the "stuff you can't see 'til you lay it out" issue the hard way.

I'm not, as I said earlier, taking sides in this argument, which I think is a no-win scenario, as some people will choose to believe that publishers are heinous, money-grubbing bastards who are hosing the consumer, and should be expending more of their filthy lucre to perfect the ebooks (although their various and sundry financial statements don't seem to be supporting that position); and then there's those just seeking info; and then there are those who've made books of all kinds, digital and paper, and have varied perspectives.

If legacy publishers are getting "full-boat" for their ebooks, then, I concur; they should be putting out the same quality of ebook that they do print book. I've noted, however, that--for what it's worth--there seems to be a larger constituency of ebook readers who complain about typos than there are print book readers (and I do know this first-hand, from our POD categories--precisely the same book, as our process is absolutely geared to single-source). So: is the problem as large as we think it is...or is it the (apparently) greater number of voices raised that make it seem so? Or is it simply numbers, in that a $2.99 ebook will sell more copies than a $12.99 hardcover, or...?

It's an interesting question.

And, just for s&g's, I've asked several clients if I may use some of their pages here, for demonstration purposes (I do not know if I will obtain permission)--these clients had scan & OCR. I'm asking them if I may post 1-2 original pages from a PDF, and the resulting RAW scanned output; one book is older--20 years-ish; the other is newer, mid-to-late 90's--and if they give me permission I'll post it. I don't think most people know what raw scanned output (other than the proofers at PD) really looks like, and I think it could be useful for everyone, at least for the OCR portion of the discussion. IF I get permission.

HTH in our gracious, genteel cogitations on the topic,
Hitch