The fight for a reflowable e-document standard

[Bob Russell]

If you've ever wondered what all the fuss is about with regard to e-book formats and the new standards, and so forth, this is your chance to get a quick look into the situation without having to spend your whole day on it. I have to admit that I haven't really paid much attention to this topic until recently, and am only beginning to get familiar with it myself. But it's an important topic, so it's about time to start getting up to speed.

My attention has been caught today by two recent shots that have been fired in the fight for a document standard. But before we address that, we need to get caught up a bit with regard to the landscape.

What are we talking about here? The driving need is for a widely used format which allows for documents to be successfully presented on a wide range of display sizes, such as handheld devices. Adobe's pdf, for example, is a widely successful document standard, but does not excel at handling presentation on multiple screen sizes. Other formats may be sufficient, but if there are a dozen of them and they are proprietary, then you still don't have any level of assurance you can use the format on future devices. Even worse, publishers could spend a fortune trying to prepare and support e-books for all those formats.

The OpenReader Camp

So, in one corner of this fight for e-readability, is David Rothman, Director of Strategy and External Relations for OpenReader. You are probably already familiar with David through the excellent TeleRead Blog. The OpenReader Consortium describes themselves as "a nonprofit organization developing open digital publication standards. We serve publishers, consumers, e-book retailers, librarians, and other key stakeholders in socially responsible ways."

David has coined the phrase "the Tower of e-Babel" to describe the current situation of incompatible e-book formats. The OpenReader site describes the problem this way: "Any Compact Disc will play on most any CD player, and the VHS vs. Betamax battle is long over. Why can’t electronic books, newspapers, journals and other publications be as easy? Consumers are confused right now. They don’t need different eyeglasses to read a paperback rather than a hardback. Why should consumers have to worry about the software they use to read downloads of electronic publications? Just in e-books alone, more than 20 different formats exist."

OpenReader has some specs in development, including a draft version of the Binder Document Specification (which appears to be an XML specification for a way to collect all the files that make up a publication), and a draft version of the Basic Content Document Specification (which appears to be an XML specification for the actual contents of a document). More than just an abstract specification, OpenReader says that "dotReader will be the first OpenReader-compatible commercial program [and] FBReader will be also be compatible." FBReader is an e-book reader for Linux PDAs that supports, for example, the Nokia 770 and the Sharp Zaurus.

The IDPF Camp

In the other corner of this fight, there is another standards group working on the tower of e-Babel, and it's the IDPF - the International Digital Publishing Forum. The IDPF describes itself this way: "The International Digital Publishing Forum (IDPF), formerly the Open eBook Forum (OeBF), is the trade and standards association for the digital publishing industry. Our members consist of academic, trade and professional publishers, hardware and software companies, digital content retailers, libraries, educational institutions, accessibility advocates and related organizations whose common goals are to advance the competitiveness and exposure of digital publishing." While I don't know if there is a public membership list, it does boast a remarkable list of members as can be gleaned from the published presentations, if nothing else. Among those worth noting are Adobe, MobiPocket, Motricity, iRex and eBook Technologies. As I don't have a full list, I'm sure I've left out many other prominent names.

As quoted from PublishersWeekly, here is another very succinct description of the problem we face. "At present, there is no common standard used by producers and manufacturers. As a result, customers can’t read a Palm e-book on a Microsoft Reader, noted Nick Bogaty, executive director of the IDPF. If companies adopt the new standards, not only will customers be able to read e-books on different devices, but e-books will be cheaper and easier to produce, which should lead to more titles being available, said Bogaty. 'We’re looking to create the MP3 for e-books,” Bogaty said about the goal of attaining file flexibility.'"

I should point out that the IDPF is much more than a standards group. Some of the guiding values are:

• Providing a forum for the discussion of issues and technologies related to electronic books.
• Developing, publishing, and maintaining common specifications relating to electronic books and promoting the successful adoption of these specifications.
• Promoting industry-wide participation of electronic publishing through training sessions, guidelines, and demonstrations of proven technology.
• Identifying, evaluating and recommending standards created by other bodies related to electronic books.
• Encouraging interoperable implementations of electronic book related systems and providing a forum for resolution of interoperability issues.
• Accommodating differences in language, culture, reading and learning styles, and individual abilities.

But the IDPF has done a remarkable job of involving the publishing industry at many levels, as well as e-book sellers. And they have come up with a draft of a container format is based on zip archive technology. It supports publishers while working on document production, distribution to the sales channels, and for delivery of the final result to the end user. It will "allow publishers to release only a single standard file into their sales and distribution channels instead of the multiple proprietary files that they currently produce. The new ZIP-based proposed standard, OEBPS Container Format (OCF), is an extension of the packaging format defined by the OpenDocument OASIS Standard (ISO/IEC 26300)."

And in addition to this container format (which is evidently analogous to the Binder format from OpenReader), there is another working group developing the document standard itself. This second specification is the next generation of OEBPS (Open eBook Publication Structure). It is an XML based e-book format from the IDPF (International Digital Publishing Forum). It is being developed to "focus on detailed control of content rendering, navigation and accessibility, and alignment with other standards efforts."

There have been announcements that OSoft will support the IDPF standard in DotReader, as well as support for these IDPF standards by Motricity (eReader, but it's not clear they've made any actual product plans to support the standards) and iRex ("We expect to support the IDPF standards for use in the iLiad e-reader in the future.")

Other Players?

Most certainly there are other efforts to develop document and container formats. And there may be more related document standards that come closer to solving these issues than I'm currently aware. But this gives a basic lay of the land as I understand it so far. And it gives you a basic framework from which to understand the two recent shots fired in the battle. Let's take a look.

Shot 1: OpenReader is the solution

David Rothman, in his article published by Publishers Weekly, points out that the only DRM content you can read on a Pepper Pad is from MobiPocket. That means you can't read anything from the Fairfax County Public Library's digital offerings, which are in pdf format.

He reminds us that the landscape is full of broken promises. "The big companies' unkept promises go back at least as far as 1998, when Microsoft and others said they would avoid a VHS/Beta–style war. Unsurprisingly, the agreement ended up a joke among the e-book cognoscenti."

He gives credit to the IDPF as the main e-book trade group, but says that the IDPF is dominated by big corporations, whose interests then dominate the organization and most publishers who "remained ignorant of the nuances of e-book formats."

In what is probably a bit of posturing, he says that the IDPF "has unintentionally stymied the popularity of e-books by failing to adopt consumer-friendly standards." What he is saying is that the IDPF has not so far created that successful and widely adopted standard that would prevent the current tower of e-babel. But as they are one of the main forces trying to solve the problem, it might be a bit much to put the blame for our current state on them unless one thinks they are a genuine obstacle to progress.

Then having established the problem, and arguing that the IDPF is not the solution, he argues that they should leave the standards business altogether. (Leaving it to OpenReader, of course.) He does argue that standards bodies like OASIS should be more involved in the standards. But there has been indication that the IDPF is interested in doing just that, and I could be wrong, but I don't believe we've seen a lot of success to this point from OpenReader on the OASIS front yet.

Basically, I think David argues that the OpenReader specs, being more technology focused rather than publishing industry focused, is producing a better spec and therefore should become the based standard for e-books. Then with adoption by standards bodies and by the publishing industry as it becomes more tech savvy, a firm foundation will be set. In his words, "As a starting point, standards setters could consider the OpenReader format, which is now in draft form. (I cofounded the standards-minded OpenReader Consortium.) With the right software features in place, the OpenReader standard will allow forums, blogs and annotations to be visible within books, including even copy-protected titles, making them competitive with the Web."

And then, of course, he points out that DRM needs to also become a part of the e-book standards. I wholeheartedly agree. The biggest obstacle for e-book compatibility is not the container or even the document format, but the DRM mechanism. Even HTML or text documents are useless if there is proprietary DRM protection on them.

So there you have it. Together with the support for their format in DotReader, the basic argument for OpenReader is given succinctly and clearly.

Probably David has done a better job so far of getting the word out to the technical crowd. He has an excellent and active blog which is a forum for presenting his ideas. And he has been very eloquent and persuasive in his arguments. But there's always another side to every issue, isn't there?

Shot 2: IDPF is the solution

While the IDPF is likely to become more and more active in publicity in the tech world as time passes, they present a more "official" face and we see more from them in places like presentations from their annual conference and press releases, and the occasional publication.

On the other hand, Bill McCoy from Adobe is a big supporter of a solution for the tower of e-Babel, and has written about the topic in places including his own Adobe blog. Let's look at what he has had to say in his latest entry on the topic.

He directly addresses David's article, and starts by setting the common ground. We all have the same end goal. It's how to achieve those goals that we dispute. "I agree with his main points - we have to make digital reading simple and compelling for end users, and the plethora of proprietary eBook formats has created consumer confusion. Where I disagree is that I see the answer at hand."

Also as a prelude, he points out that we have a successful and widely used format for e-books that are fixed format and page-oriented. "[F]or paginated final-form 'ePaper' there is no Tower of e-Babel - PDF is the answer, it's game over." It's hard to argue that point, no matter how frustratingly slow it may be to get Acrobat Reader to load in order to read a pdf document. I know that in many cases, I avoid following a link to a .pdf document because it's often not worth the wait. I know others feel the same way because there are often warning labels on links to alert the reader that they will end up waiting on a pdf document. But, imperfect as it may be, it gets the job done, and most importantly it's almost universally adopted. As Bill says, "Game over."

So how should we address the reflow problem for various sizes of displays? Far from considering the IDPF an obstacle to progress, Bill says that "this problem is now being rapidly and effectively addressed by the IDPF. Publishers, vendors (including Adobe), and library and educational groups have come together in IDPF to create the open standards that will end the Tower of eBabel."

While David Rothman's criticism of the IDPF hinged on it's dominance by publisher interests and big companies, together with the lack of a firm technical foundation, Bill McCoy suggests that David's alternative is troubling. I.e. to let OpenReader do all the technical specifications and then get them "rubber-stamped" by OASIS without effectively bringing the publishing industry into the picture.

Bill doesn't pull any punches. He says of David's recommendations that, "Well, I think this is elitist, egocentric nonsense that hasn't a prayer of working. No doubt it's messier to have publishers, competing vendors, and nonprofits all coming together. True, not every publisher has industry-leading technologists on staff. But many do, and this kind of participative process is part of building successful (by which I mean broadly adopted) solutions." He suggests that much of the battle comes down to egos. And he goes on to point out that e-book document standards are not likely to be much of a priority for large standards bodies like OASIS anyway.

Bill basically concludes that the IDPF is a very effective organization to address the tower of e-Babel. And that, in fact, it's the best approach we have. "I believe that we're all a lot better off with publishing industry working together in an architecture of participation, and IDPF is where this is happening now. The results of this, including the IDPF OCF container format standard, are already bearing fruit and the Tower of e-Babel that plagues reflow-centric eBook formats will start falling down over the coming months. There's always room for more contributors in the IDPF."

Conclusions

What we have seen is just a snapshot of the battle for e-book standards. As I said in the beginning, I'm not an expert. But I do feel this is an important discussion, and one that has previously been fairly inaccessible and incomprehensible to most people interested in e-books.

If you expect me to choose a side, then you will be disappointed. My greatest desire and hope in writing this article is that it will prove helpful in making this whole topic more inclusive, by making it more accessible and interesting to a much broader audience.

Where I have incompletely or inaccurately represented any of the positions, there is always room for further discussion and input. But what we should all remember is that, outside of a few special interests, we all want the same end goal of compatible e-books. The people fighting for that goal are our heroes. And one way or another, the e-book industry is going to explode. It's going to be an exciting ride, and a lot of fun to watch!

Update:As this is published, I notice that there's another response over at TeleBlog to Bill McCoy's comment. I haven't read it yet, but as I indicated, this is an ongoing discussion, and I hope that now you are in a better position to enjoy the battle.

[rlauzon]

The problem with IDPF is that whatever solution they come up with will be a closed, proprietary solution - regardless of what they call it. Their "solution" will still have the same issues as any other closed, proprietary solution: if they go away, you are screwed. If their format fails, any books you have become worthless.

We already have open eBook file formats: RTF, HTML - to name a couple. We don't need IDPF.

Also, to use an analogy, anyone can create a DVD player. But if you want to play comercial DVDs, you need to get a CSS license. To get that license, you need to sign a contract that you will put deliberate defects into your DVD player that will prevent users from exercising their rights and enforce rights that the content holders do not legally have.

IDPF is simply the CSS for eBooks.

OpenReader's problem is DRM. "Open Source DRM" cannot exist. Right now, DRM, as someone put it, is like putting the content in a safe, putting the safe into the criminal's house and telling the criminal to not crack the safe. "Open Source DRM" is the same, except that the combination is written on the bottom of the safe.

Because most authors are not computer-savvy, they believe what their publishers and groups like IDPF tell them - mainly that they need to "protect" their content from "pirates" and use DRM (even though DRM doesn't do a darn thing to protect their content from pirates). So without DRM, OpenReader is doomed to fail - unless something changes.

IHMO, the whole "Tower of eBabel" is based on the fact that the Content Cartels want to own content forever. They have no incentive to create an open eBook standard - they want you to pay for your content again and again.

It's going to take successes like Scott Siegler and Cory who post their content for free - but who are also successful in the commercial market as well. Baen seems to be doing very well with DRM-free eBooks (the last 2 issues of Baen's Universe were available in RTF - and easily convertable to my iLiad).

It won't be IDPF that throws down the Tower of eBabel - they are doing nothing but shore it up. OpenReader may do something - but right now, it's yet another open format added to the list of many open formats and doesn't have any great benefit over any other open format.

It will be us, the readers, that will throw the Tower down, by not buying eBooks in closed, proprietary, DRM-laden formats.

[davidrothman]

Hi, Bob. Major thanks for caring about the e-book standards issues. Here’s to the end of the Tower of eBabel! Meanwhile, as an OpenReader co-founder, let me add to the fun and supply a few more details.

First off, for those who didn’t see my uppity essay in Publishers Weekly, let me repeat your pointer to the following URL:

http://publishersweekly.com/article/...l?text=rothman

Angered by OpenReader’s presence in PW, Bill McCoy has erupted with a surrealistic response. It’s just what we’d expect from an Adobe executive who also sits on the board of the International Digital Publishing Forum. And I’d like to counter one McCoy canard right now--Bill’s perfect silliness about the nature of our standards-setting.

Contrary to his establishmentarian line, OpenReader wants a by-the-book approach with far more integrity than the IDPF has shown.

Specifics? Well, unlike the IDPF, we strongly prefer to see the actual standards work spun off to an OASIS technical committee. We’re working on this and want it done in style, one reason we haven’t plunged in immediately. Bill meanwhile has helped our cause with insulting remarks challenging OASIS’s suitability for e-book standard-setting. Thanks, buddy!

Let me also say we believe that membership selection and other matters should be determined by OASIS rules, not rules from OpenReader. Also, someone other than Jon Noring, founder of OpenReader, should chair the OASIS standards committee--the man has suffered enough abuse already from the likes of Bill. Meanwhile, yes, Jon has come up with a draft spec. But we think that the development and evolution of the standards should be substantively handled elsewhere with people from many organizations involved, including, yes, Adobe and ETI.

We just don't want Adobe and ETI dominating the show. While publishing interests may sit in on the IDPF technical meetings, the usual suspects are making the real decisions, which is scary when you think about the potential use of proprietary DRM to nullify "open standards." As I see it, the OASIS committee should and will be far more than just the rubber stamp that Bill McCoy wants the IDPF to be for Adobe and the rest. OASIS, after all, is the outfit which popularized the Open Document Format, and which includes such members as Sun and IBM, along with people passionate about e-books.

Now, compare OASIS to the tiny world of the IDPF. It may be a Player in e-books, but by OASIS technical standards, we’re talking about a midget. Reps from ETI chair both the container committee and the core format committee of the IDPF--frog ponds lacking the widest range of technical talent. Better that standard-setting happen in the OASIS sea. Otherwise a year or so from now, MobileRead will still be carrying complaints from eBabel victims. As OpenReader views matters, the more neutral venue of OASIS would be a far better setting than the IDPF for the razing of the Tower.

Something else to clarify is our attitude toward publishers; we'd actually like to see more publisher power in the IDPF, most of whose board members are now from tech-related companies rather than traditional publishers or librarydom. While big publishers have generally been more influential in the IDPF than smaller ones, even the giants are feeling the pain of supporting all those formats at the consumer level.

For years Jon and other techies tried to get the IDPF to listen to complaints from publishers and others. It was deaf, and that's why OpenReader came about--followed by the IDPF's naked attempts to steal the standards spotlight. The e-book establishment’s strategies can only go so far, however. Just wait until the topic of DRM comes up, and usual suspects either run off in different directions or settle the matter in a way that's more helpful to present business models than to publishers and consumers. Microsoft, while an IDPF member, isn’t actively participating in the standards process; and whatever the reason, a leading e-book-retailer isn’t even listed as an IDPF member any more.

I know I'm skeptical about the IDPF, but there are definite reasons. Since the late '90s this group has failed to live up to its promises despite all the hundreds of thousands and perhaps more than the group has spent (that's why some large publishers privately regard the group as a joke, keeping their sentiments to themselves since they’re too busy putting out books to challenge the likes of Bill). At one point the IDPF even said it would be “agnostic” about consumer standards--a wonderful euphemism for a do-nothing approach.

By contrast, with virtually $0 for OpenReader and a lot of starving at OSoft, an OpenReader-compatible dotReader will soon be appearing. Go to dotreader.com and share with OSoft your suggestions. Rather than just dissing IDPF, OpenReader has been hard at work solving the eBabel problem, and we hope other implementers will join OSoft.

We’d love for an OpenReader version of dotReader to be out right now, but I don’t think we’ve done too badly, especially considering all the bizarre statements that we’ve had to spend time countering from IDPF-linked people. Bill McCoy’s latest blog entry is a good example. So is the weird stuff that MobileRead quite unwittingly picked up from him and perhaps those of a similar mindset.

Bob, you can help the cause of open e-book standards by continuing to study the issues yourself rather than just quoting Bill. Sometimes one side is right or at least mainly right. To name an example beyond the facts already given, I’m vastly amused by Adobe’s efforts to portray a OpenReader as an elitist techie conspiracy. We’re actually coming at this from the opposite direction; we want e-books to be simpler to for consumers to use, and that means a universal format capable of handling many applications. It’s true that Jon Noring, OpenReader’s founder, is accomplished enough technically to point out some major technical flaws in the IDPF’s standards work, but if anything, I’d love to see more of THAT kind of elitism. It’s why I want the standards efforts to be done by top experts at OASIS, under close supervision from publishers and ideally with far more active participation from them--especially smaller houses, and even individual authors.

Amusingly, Bill McCoy’s reply to my Publishers Weekly article didn’t even mention Jon by name (although you can bet Bill has in other places). Yes, I wrote the article; but Jon founded OpenReader. Am I the master uber-techie Bill fears? While I’ve spent years writing about e-book issues, it’s been from the perspective of a content guy. I’m not a coder but have absolute respect for the true whizzes who can bring to life my hopes for e-books. I’m author of six p-books from Ballantine, St. Martin’s Press, etc., and I’d love to see e-books more competitive with the Web. I can’t tell you how much interactivity would help. We’ve seen the possibilities from the blog world. That is why I hate it when Bill McCoy and friends try to get in the way. I could counter more nonsense from the Bill, Adobe and the IDPF; but I’ve already spent enough of my weekend.

As for the TeleBlog (http://www.teleread.org/blog), it’s about much more than e-book standards--everything from the $100 laptop project to libraries and copyright and public domain projects. I hope people will drop by while continuing to enjoy MobileRead as much as I do. In case people are curious, the TeleRead site as a whole is devoted to the cause of well-stocked national digital library systems in the States and elsewhere--it existed along before OpenReader did. TeleRead.org dates back to the 1990s.

Now on to Ron’s comments about DRM. I’d love for people to tell publishers about their hostility to the technology, and for e-book reviewers to note when a book is DRMed and what the conditions are. Let the marketplace prevail! Alas, however, without DRM available, major publishers will avoid any standard. So OpenReader intends to oblige them and deal in good faith, while, however, alerting publishers about the downside. See Jon’s essay called The Perils of DRM Overkill for Large Publishers. The URL is:

http://www.teleread.org/publishersdrm.htm

Ron, we’re with you--I myself try to avoid today’s DRMed books as much as possible, given the very real chance that future hardware may no longer be able to display them. Let’s hope the publishing industry can see the light. If nothing else, meantime, remember that any OpenReader-related DRM will be (1) optional for publishers, (2) done in a way that avoids an Adobe- or Microsoft-style chokehold and (3) allow books to be accessed permanently and owned for real. With DRM, OpenReader will go either the Open Source route or a semi-proprietary one, if the standards setters believe this is necessary. I’m rooting for the open source possibilities if it can be user friendly. We’ll see what the standards setters deem effective without anything being a chokehold (we know that today’s DRM is better at enforcing Adobe’s marketshare than in protecting e-books).

Meanwhile, Ron, I hope you’ll show some optimism here and spread word about OpenReader’s possibilities. Despite no budget for marketing, we’re in touch with some of the world’s largest publishers, and they’re excited by our technology, as well as dotReader’s DRM--which is far, far gentler on readers than the present variety from Adobe and the like. Just as encouragingly, smaller publishers, stores and distributors are telling us how much they hate the costs and hassles of the present DRM systems. Yes, publishers are already starting to commit to us in Real Life. Freeload Press, subject of news stories the world over, will be using OSoft’s dotReader next year because of many students’ aversion to PDF textbooks. Adobe can talk all it wants about the future. OpenReader and its implementer will unveil a reflowable solution in just weeks, one that should make students more comfortable with e-books and give them more benefits such as shared annotations. As for formats, we’ll have wrinkles that go far beyond the existing HTMLish approaches.

Stay tuned, Ron, and keep the feedback coming. Unlike the IDPF suits, you’re looking out for the interests of readers, and we appreciate that.

Thanks,
David

David Rothman,

Co-founder, OpenReader Consortium
http://www.openreader.org | davidrothman@openreader.org
Telephone: 703-370-6540 (near Washington, D.C.)

Main perp of the TeleRead Blog: http://www.teleread.org/blog

P.S. Detail for Bob: My Publishers Weekly piece listed some Fairfax County Public Library books that weren’t in Mobipocket format, the only DRMed one that the Pepper Pad handles. But FCPL does carry Mobipocket editions of other books. You’d have been better off without the comma after “offerings.” Hey, not to worry. Your main point stands. Why should e-book formats determine which books we can read from the library? Thanks again for your interest in eBabel matters!

I've never seen open source on any major technology work out for consumers. I know folks like to point to Linux and show how successful it is. But unless you are in IP security or you are a computer geek, you probably couldn't run Linux. It simply is not popular among the average computer consumers. To think that any open source, non-publishing industry eBook standard will ever take off and be a major benefit to consumers is pure fantasy. It will simply be just another of the 30+ standards already in existence.

The problem isn't coming up with an eBook standard. We already have a pretty good one that the publishers support. The problem is getting companies like Microsoft, Adobe, MobiPocket, et cetera to not force consumers to use their particular interpretation, that is proprietary software, to read eBooks. These companies make money off of selling their software to companies and individuals to create eBooks that are then readable only with their respective readers.

What is missing, as far as I can tell, is a powerful documentation suite capable of producing multiple versions of the same document. For example, a program like FrameMaker for producing eBooks. The problem is, companies like MobiPocket, Microsoft, and eBook Studio or not going to share their proprietary formats with any company that wants to develop an application to create multiformatted eBooks. It will cut into their profit.

So what really needs to happen is that the publishing industry needs to develop an eBook standard, including encryption methodology and copyright protection, and then develop the software to publish eBooks in this standard. The publishing industry, afterall, is their industry. It is up to them to unite and take the bull by the horns. No software company will ever be able to pull this off, either alone or through a consortium. It must originate from, and be controlled by the big publishing giants working together. Little mom and pop publishing houses will have no impact on the industry. They simply don't sell enough books.

Bottomline is that I don't think this will happen anytime soon, if ever. I think we will continue to see dozens of competing products and formats unless the publishing giants wake up and realize just how much money they can make selling eBooks. I suspect they will ignore eBooks until it is almost too late, much like Kodak and many other film manufacturers ignored digital photography until they were nearly bankrupt. The time will probably come when consumers will demand eBooks. Until then expect a very slow adoption by an industry that has its roots planted over six centuries deep in ink and paper publishing.

To reiterate my main point: Standardization of eBooks publishing cannot, will not, must not initiate at the hands of those who will not do the publishing. It must and will initiate from the publishing giants, or it will not initiate at all.

[davidrothman]

"The problem is, companies like MobiPocket, Microsoft, and eBook Studio or not going to share their proprietary formats with any company that wants to develop an application to create multiformatted eBooks. It will cut into their profit."

Amen! Many thanks for a wonderful example of the harm from the Tower of eBabel. Of course, OpenReader will reduce the need for a zillion and one formats--given its usefulness in a number of situations. Try running PDF on a PDA, for example, particularly if the files are DRMed and you're stuck with the pathetic Adobe Reader. Many and probably most e-books are NOT tagged for PDAs. In practice this is a rather theoretical solution.

By contrast, OSoft's dotReader (dotReader.com), the first dotReader-compatible software, will work great with OpenReader-format files on most anything from PDAs and cellphones to desktops with 32-inch monitors.

"Standardization of eBooks publishing cannot, will not, must not initiate at the hands of those who will not do the publishing."

Bingo! Exactly the problem with the IDPF. It's been FAR more responsive to the needs of Adobe, etc., than to the needs of big publishers demanding a robust consumer standard. Meanwhile smaller publishing houses, which also count, have suffered the financial consequences of the Tower of eBabel even more than the big boys have.

"The problem isn't coming up with an eBook standard. We already have a pretty good one that the publishers support."

Huh? Not at the consumer level. As for the production level, things have a way to go--yes, even there.

"It will simply be just another of the 30+ standards already in existence."

You might want to reconsider. Jon Noring, founder of OpenReader, has been in the thick of the standards action for years and knows the weaknesses of the existing production standard. Plus, he's a very small publisher himself and, amusingly, is even an invited expert to the IDPF. Makes sense. He's like a priest doing nightclub duty to reach sinners. Along the way, he's looking for ways to harmonize OpenReader with certain elements of the IDPF's efforts. In the other direction, the IDPF has benefited from Jon's major technical expertise--a lot more than it deserves.

Meanwhile OpenReader's first implementer, OSoft, is coming out with a powerful implementation in dotReader that runs not just in linux but on a bunch of other platforms, including Windows. So dotReader's reach will be far, far behind that of the usual open source software.

As noted, look for OSoft's dotReader in weeks. The main reason for the delays have not been technical but simply the usual business challenges that small companies face. It does not help when a bully like Adobe so aggressively spreads malicious misstatements about the related standard; but actually that will be pretty useful in the end. Some real proof of dotReader's viability, and that of the OpenReader standard, will be in the quality of this first implementation and new features not found elsewhere. At that point, publishers more than ever will understand why a giant like Adobe has spent so much time attacking the OpenReader standard.

The ultimate proof of viability will of course come in the form of adoptions. As noted, Freeload Press, subject of major international publicity, is already planning to use OpenReader in the near future, and meanwhile we're oh so close at some publishing giants impressed with dotReader's feature set.

I know. Publishing is a conservative industry. But dotReader, in areas ranging from interbook linking to suitability for E Ink machines, is so far ahead of the competition that large houses will find it risky NOT to embrace the OpenReader standard. While dotReader can deal with a number of formats, OpenReader will be the preferred one because of its interactivity and for other technically related reasons.

Thanks,
David

(Running out of time but enjoying the chance to enlightened folks about e-book standards and the considerable potential of our first implementation.)

[rlauzon]

Originally Posted by davidrothman

If nothing else, meantime, remember that any OpenReader-related DRM will be (1) optional for publishers, (2) done in a way that avoids an Adobe- or Microsoft-style chokehold and (3) allow books to be accessed permanently and owned for real. With DRM, OpenReader will go either the Open Source route or a semi-proprietary one, if the standards setters believe this is necessary. I’m rooting for the open source possibilities if it can be user friendly. We’ll see what the standards setters deem effective without anything being a chokehold (we know that today’s DRM is better at enforcing Adobe’s marketshare than in protecting e-books).

But this type of DRM cannot exist.

The stated purpose of DRM is to lock the content to the purchaser to prevent piracy. DRM with the attributes you describe cannot do that, for the simple reason that the lock design and key are available for anyone to see and use. It's like putting a lock on your house and then always keeping a spare key under the door mat. There's no security.

5 minutes after any open source DRM is created, someone will have written a program to take the DRM off. Make it 10 minutes if they go the semi-proprietary route.

If there is no security, why bother with putting DRM on in the first place? It's adding cost to something and getting nothing in return.

If we bend reality a bit and assume that such a DRM scheme can be made to work, it still has all the problems of any other DRM scheme:
+ If support for the format goes away, your content becomes useless.
+ You cannot transfer your content to someone else.
+ If a device comes out that does not support the format, you cannot read the eBook on that device.

Simply put, your DRM-scheme cannot be "done in a way that avoids an Adobe- or Microsoft-style chokehold" and cannot "allow books to be accessed permanently and owned for real." and still be DRM. We either have no DRM, or we have a chokehold and eBooks cannot be accessed forever.

And all this doesn't even cover what the true purpose of DRM is (i.e. format/device lock-in). Which means that it will be a tough sell to companies like Sony.

[segatang]

I think the problem of ebook standard is hard.

For example, converting music CD to mp3 is easy.
The users think the music is still the music.

Converting book to ebook is not easy.
The format and layout of book will lose without careful re-edit.

The users find the readibility is somewhat different due to the hardware or ebook itself.

It is hard to rebuild the 'feeling of reading book' of magzine or journal paper for a small size ebook hardware, but 'listening to music' is the same in any mp3 player...

[Jorgen]

What the world needs is less people making standards.

[davidrothman]

Replies follow to Ron, Segatang and Jorgen.

RON: There are plenty of other ways in which OpenReader will stand apart from other formats--this transcends the DRM issue. Meanwhile let the standard setters work the DRM out.

My personal *preference* would be for a nonproprietary solution. But another solution would be a semi-proprietary system with many different organizations involved, so as to avoid the possible security problems of a fully open approach, while at the same time avoiding Microsoft II or Adobe II chokeholds.

All kinds of companies within reason could have access to the DRM technology to one extent or another, as long as they signed NDAs, etc., and meanwhile no restrictions would exist on the nonDRMed format.

There could also be limits on fees charged so as to keep the DRM costs a fraction of what they are now.

Beyond security, bear in mind that the big DRM issues are usability and continued access, just as you seem to think; and both are on our radar screens. We enjoy close relations with the library world and have some very specific ideas in terms of making DRMed books forever accessible to their owners.

One of OpenReader's founders is even an archival consultant who formerly oversaw record-keeping via his job as information services manager at the World Bank.

As for ownership transfers, yes, we care about that topic as well, and if I have my druthers, OpenReader DRM will allow that option to be enabled. If it isn't, then book prices should reflect this.

Now--about future devices reading OpenReader format books that are DRMed. Our first implementer, OSoft, is fervently committed to a crossplatform philosophy. And we'll encourage other implementers to feel the same way. The fact that the core format is nonproprietary and that we want to avoid a DRM chokehold should help.

Ron, I can see that DRM is your major issue, and if that's the case, I'd suggest you keep in mind our personal hostility toward the technology.

Also, remember our eagerness to let publishers avoid DRM, while at the same time having better, gentler DRM for pubishers insisting on it.

Follow the pointer to Jon's piece on DRM overkill. With scanning of p-books as an option for pirates, DRM, as we see it, is rather silly as a rule. But big publishers disagree.

And so, without DRM, there's no hope of an e-book standard that the big publishers will go for, alas, and that's of big concern to me since they hold the rights to so many modern classics.

Meanwhile, at least with OpenReader around, you'll *know* that DRM will be just an option for publishers. With the proprietary formatters, you don't know what the future will hold--whereas with the OpenReader scenario your "vote DRM out" approach would work just fine if that's what users wanted.

Let the market prevail. Some users may tolerate DRMed books but insist on paying less for them. In fact, as I recall, an unscientific TeleRead poll seemed to suggest that many would be interested in those terms. (See http://www.teleread.org/blog/?p=5369.)

Then again, publishers may find that the tolerated prices of the DRMed versions will be so low that they might as well drop DRM. Hard to say.

Meanwhile OpenReader is committed to offering different options.

SEGATANG: You write: "The format and layout of book will lose without careful re-edit."

Actually dotReader, our first implementer, can work with an image-based approach. But that's inappropriate, as we see it, in the overwhelming majority of cases.

Here's to reflowability and readability! Why replicate the present negatives of PDF?

JORGEN: "What the world needs is less people making standards." Exactly. Let's have trains run on different gauges and electricity being AC or DC, depending on whether in your home or your neighbors. That's about the way the world would be if the IDPF did standard setting in those cases. Since we prefer a single gauge for railroad tracks in a typical case, etc.--well, the e-book equivalent--OpenReader exists.

Thanks,
David

[b_k]

Originally Posted by rlauzon

But this type of DRM cannot exist.

The stated purpose of DRM is to lock the content to the purchaser to prevent piracy. DRM with the attributes you describe cannot do that, for the simple reason that the lock design and key are available for anyone to see and use. It's like putting a lock on your house and then always keeping a spare key under the door mat. There's no security.

5 minutes after any open source DRM is created, someone will have written a program to take the DRM off. Make it 10 minutes if they go the semi-proprietary route.

If there is no security, why bother with putting DRM on in the first place? It's adding cost to something and getting nothing in return.

So if I would think like you, PGP must be insecure cause the algoryhtms are well known and open-source. Anyone knows the times it would take to crack PGP-keys with a (as of now) secure algorythm? If I remember right, it would take quite a long time.

If we bend reality a bit and assume that such a DRM scheme can be made to work, it still has all the problems of any other DRM scheme:
+ If support for the format goes away, your content becomes useless.
+ You cannot transfer your content to someone else.
+ If a device comes out that does not support the format, you cannot read the eBook on that device.

Simply put, your DRM-scheme cannot be "done in a way that avoids an Adobe- or Microsoft-style chokehold" and cannot "allow books to be accessed permanently and owned for real." and still be DRM. We either have no DRM, or we have a chokehold and eBooks cannot be accessed forever.

And all this doesn't even cover what the true purpose of DRM is (i.e. format/device lock-in). Which means that it will be a tough sell to companies like Sony.

For the "how we treat customers"-thing I think the same way.
But I think it would not be impossible to develop a open source (=portable), user-friendly (=resell the drm-ed content) and secure DRM. To bad that is simply not what the book/drm-software publishers want.

I as customer would like to see a fair and portable solution which does not lock me to one device/os/vendor, but again thats not what the industry wants.

[yvanleterrible]

MP3 is not perfect but has been adopted by and is the standard for every mahine. For e-books every major company is pulling the blanket their way, now it's crumpled up in the middle of the bed, useless.

I'd be in favor of a total PDF machine with Acrobat in ROM installed on board. That could take care of every other format. And if you want to write, for a bigger fee, a machine set up with Acrobat Pro and foldable keyboard will provide ample performance. Acrobat is complete as it is and does not need that much functional improvement, since an e-book reader will be considered expendable like an MP3 player, the latest version would suffice for the life of the reader. It is naïve to build such a possibly inexpensive machine with expandability or updates in mind, because it's a throwaway tool like so many supposedly durable predecessors, and it drives up the price way past what the market needs to begin . Building a totally new engine(CPU) and writing new software is illogical and drives up cost needlessly. All we want after all is a smooth performing reader with e-ink. I drive a Toyota not a Porshe, I would not want one. I am sure a rewritten Acrobat(unix ?) would work properly on a smaller processor.

Case in point. We all have a computer. How many versions of software for an app. have you installed for that particular machine? Every machine has been rendered obsolete or heavily taxed by subsequent operating system versions. This computer I'm writing on has seen Windows 98 and is barely dragging through XP. It will die with Vista. Software and hardware should be born and should die together and have an efficient and long operational life.

The way things are going now, the day is far where e-ink will replace paper. The e-book reader is being politicized into oblivion. Let's stop talking so we can read, preferably under comfortable motionless blankets ...

[davidrothman]

"All we want after all is a smooth performing reader with e-ink."

The E Ink area, in fact, will very likely be one of dotReader's strengths, if you go by its resource-frugal nature and care about performance. Plus, the OpenReader format will offer interactivity, interbook linking and other fun stuff. - David

[watcha]

Why can’t DRM technology be programmed to reflect real life? You know, five years after having downloaded that award-winning novel you should be able to sell it on for a quarter of the price. Ten dog-eared years later, you should be able to donate it to charity. If it was a real bummer – ie hardly sold any downloads after critics slated it - you should be able to put it in a box with 20 others at your garage sale for mere pence (OK, put in on e-bay).
And you should be able to lend it to family and friends (OK, allow - as is now possible – for it to be installed on six different machines).

Me, I’m just an old-fashioned reader fed up with carrying heavy volumes with me all the time. I’m all for this open-book architecture lark while realising the labourer (artist/author) is worthy of his hire. Yes, I’ll pay so they can have their royalties…but why should I continually line the pockets of software companies after initially paying upfront for their reader?

[rlauzon]

Originally Posted by segatang

For example, converting music CD to mp3 is easy.
The users think the music is still the music.

Converting book to ebook is not easy.
The format and layout of book will lose without careful re-edit.

I think that you've confused the conversion of the content into a usable format, with the creation of the actual content.

Yes, conversion from CD to MP3 is easy because someone did the hard work upfront putting the music into a digital format.

Converting a pBook into an eBook is definately not easy, because the hard work is putting the text into a digital format. But if we assume that most authors use word processors today, then converting that into an eBook is so simple that a child can do it - if we know what format we want the eBook in. So the hard part is not in creating digital content - it's deciding what format it should be in.

Originally Posted by segatang

The users find the readibility is somewhat different due to the hardware or ebook itself.

It is hard to rebuild the 'feeling of reading book' of magzine or journal paper for a small size ebook hardware, but 'listening to music' is the same in any mp3 player...

Depending on the content, words are words - just like music is music. Some content (like books that contain many code examples or diagrams, or most magazines) do need to be much edited for smaller screens, but if they create the content with this in mind in the first place, it's not that much work.

But when people talk about eBooks, we are talking about books that are almost all text and do not need to be reformated for different screen sizes. The look of the text is almost irrelevant - it's the words (and the order in which they occur) that have value.

[rlauzon]

Originally Posted by davidrothman

There are plenty of other ways in which OpenReader will stand apart from other formats--this transcends the DRM issue. Meanwhile let the standard setters work the DRM out.

So you keep saying, but I don't see it. OpenReader format is just XML, which itself is just another data format.

I'm not saying that OpenReader is bad, but I fail to see where it offers anything above what we have today.

Originally Posted by davidrothman

There could also be limits on fees charged so as to keep the DRM costs a fraction of what they are now.

(X + Y % of X) is still > X.

So DRM will always cost more and offer less.

Originally Posted by davidrothman

Beyond security, bear in mind that the big DRM issues are usability and continued access, just as you seem to think; and both are on our radar screens. We enjoy close relations with the library world and have some very specific ideas in terms of making DRMed books forever accessible to their owners.

Then you are working toward an unobtainable goal. The ideas of DRM are not compatible with accessing books forever. You cannot have a locked book that is forever accessable.

What's copyright length now? 75 years past author's death? How many of us have files that go back even 30 years? One company that I used to work for had a room filled with tapes (9 track!) and removable disk packs, that they had to keep around because of data retention laws. But they had no hardware that could access any of the data on these media. Even NASA is having issues with being able to read the old data.

Are you telling me that you are going to create a data format that, when the book finally goes out of copyright (assuming that the Content Cartel allow that) in 100+ years that someone will be able to unlock the DRM and read it? When even NASA can't keep their data formats (with no DRM on them) up?

Originally Posted by davidrothman

Ron, I can see that DRM is your major issue, and if that's the case, I'd suggest you keep in mind our personal hostility toward the technology.

I am, and I am trying to tell you that you are working toward something that is not obtainable.

Current, my major issue is that content providers (like eReader) demand that I pay hardcover price for an eBook that is locked up effectively forever, gives me no guarantee that I can read that book next year, and does not permit me to pass that eBook on to someone else.

In the future, my issue is that once the book finally falls into the Public Domain that there will be no non-DRMed version and that the book will continue to be locked even though the content is Public Domain.

I simply don't see any way to provide DRM (Well, DRM that can actually protect the content) and keep our rights as well.

Originally Posted by davidrothman

Also, remember our eagerness to let publishers avoid DRM, while at the same time having better, gentler DRM for pubishers insisting on it.

There's no such thing. Either there is DRM and it is not kind and gentle. Or you have something that is kind and gentle, but so week it's a joke DRM (in which case, why have it?).

Originally Posted by davidrothman

And so, without DRM, there's no hope of an e-book standard that the big publishers will go for, alas, and that's of big concern to me since they hold the rights to so many modern classics.

Baen disagrees and the reason that the Content Cartel holds so many modern classics is because our copyright laws are out of whack with reality - but that's another topic.

Originally Posted by davidrothman

Meanwhile, at least with OpenReader around, you'll *know* that DRM will be just an option for publishers.

Just as it is for any other format.

Originally Posted by davidrothman

With the proprietary formatters, you don't know what the future will hold--whereas with the OpenReader scenario your "vote DRM out" approach would work just fine if that's what users wanted.

But I already do. I don't buy from eReader. I buy from Fictionwise, and only those titles with no DRM.