A nexus in eBook formats ?

[SerialAeon]

Hello,
I'm "doing" computer stuffs since 15 years now I got tired of the endless format wars: pictures, videos, and now... eBooks. There is almost a dozen actively fighting formats for now, almost all platform-specifics. My question is the following: in my mind, an ideal solution would be to store the eBooks in a simple, open format that store the text, the layout (titles, chapters, sections, etc.) and any possible meta-data (type of book, etc.). Then, to offer a conversion pipeline from/to this 'nexus' format to any other commercial/proprietary/(add any wanted adjective) format.
For now, when looking at repositories like Project Gutemberg, this nexus format seems to be ... plain text, which is not very satisfying. As usual, this kind of project started when nobody cared about metadata or text layout, and now it will be a pain (according to my experience; I'm maybe wrong ?) to propose them on such a big repository.

My question is: does an ideal universal format exists ? As a biologist, I know by experience that the Darwin theory of evolution will (again) apply, and that in few years only one or two format will stay, all the other being dropped in the trash can of evolution. These one or two remaining formats will then become the de-facto universal ones. But I'm interested by the opinion of guru here that certainly know much more than me on this subject.

Aurelien

[TadW]

The OEBPS Open eBook framework could fit in this category:

http://www.openebook.org/oebps/history.htm

Microsoft for instance is using it for its .LIT format.

[Antoine of MMM]

Well, one could just use the existing CSS/HTML combo and make it happen too. I think that would bode not only better for users (imagine a similar look and feel), but then creators would really have a better control of multiple formats via a one shot deal.

Check out this site: http://www.alistapart.com/articles/boom

While I personally wouldnt care for the step that takes it into PDF format, the fact that CSS/HTML already allows for this makes for a neat one step solution for those looking to publish electronically, as well as in print.

[bowerbird]

i've developed a format called "zen markup language"
-- z.m.l. for short -- that will transform raw-ascii e-texts
like those from project gutenberg into powerful e-books.

i'll be revealing it soon.

meanwhile, you can look at john gruber's "markdown"
-- http://www.daringfireball.net/projects/ -- to see
a similar form of non-markup markup. markdown is
interesting because it bridges to xhtml, whereas z.m.l.
is meant as a standalone format displayed by a viewer
that i have programmed.

-bowerbird

[yvanleterrible]

Believe it or not Photoshop was first created to be such a beast...somewhere...far away...in some distant past...not on an other planet...

[Steven Lyle Jordan]

I believe HTML covers that need right now... even more conveniently, HTML is now used to readily convert into a number of other e-book formats.

[geekraver]

TeX/LaTeX has been around for many years, and is powerful enough to represent pretty much anything.

[ath]

Originally Posted by SerialAeon

My question is: does an ideal universal format exists?

The answer is, clearly, no. At least, not until 'ideal' has been suitably defined.

The Text Encoding Initiative is mainly for scholarly uses, and that may seem off-putting. It's mainly a base, suitable for novels and other types of texts, to which special markup (line and page identification, speaker identification, emendations, etc.) added on top.

Early versions were based on SGML, but I believe there is (or has been) an effort to convert to XML. There's no major difference on the markup level, though I believe there are on the DTD-level.

Check out their Guidelines -- and be sure to start with TEI Lite. But you *do* need an XML environment to work in, and particularly one that does check your files for markup errors.

If you want to have a more extensive example of a marked-up file than those that appear in the TEI Lite Introduction, try the Oxford Text Archive: their preferred formats are TEI-based. Their web site seems heavily frame-based, so I won't give a link, but you may look for works by Anthony Trollope -- I've just verified that the first one listed (Ayala's Angel) uses TEI Lite markup.

[Studio717]

I'd say HTML, too, since it's fairly platform independent. PDFs are also readable by most platforms that I'm aware of (perhaps not on all the smaller devices, but there is a PDF reader for my Palm), but PDFs are much more restrictive, ime, than HTML docs.

(For years I bought books in Mobipocket format and now regret it. I'm hoping to be able to find a Mobi -> HTML converter. Keyword there is 'hoping.')

[mikecook]

I know it's been a while since this thread was active but I wanted to ask another question that relates quite closely to this.

I've been researching theses master formats for sometime and it does seem that TEI, OEBPS, XML, BookX could be good contenders...deciding which to use is not that easy.

Publishers such as O'Reilly use DocBook for their technical manuals, but does anyone know if there is standard format that is used for novels (fiction, non-fiction, etc)?

Thanks,

~mike

[mikecook]

REMOVED...

I lost my connection when I originally posted but when I look to see if it had worked it didn't show up here, so I rewrote it again. My appologies

[Robert Marquard]

In this case it is really easy. OEBPS Open eBook framework. Best the currently developed 2.0 version. It defines an .epub format which is a Zip containing HTML with supporting files in XML. Mobipocket already uses it as source format. LIT does the same (using the older format).
If fact most of the ebook formats are simply repackaged XHTML.

[JSWolf]

Originally Posted by Robert Marquard

In this case it is really easy. OEBPS Open eBook framework. Best the currently developed 2.0 version. It defines an .epub format which is a Zip containing HTML with supporting files in XML. Mobipocket already uses it as source format. LIT does the same (using the older format).
If fact most of the ebook formats are simply repackaged XHTML.

The problem is not the that OEBPS is out there, but the problem is the overlaying format. My Sony Reader cannot handle Mobipocket or LIT (without conversion) for example. So it would not matter if the books were hand typed. What we need is a format whereby we can purchase a book and then convert it to whatever program or device we are using at the time so it won't become obsolete. For example, you have a palm, you buy books for this palm. All is well. The palm breaks. You have some books left that you have not yet finished reading. You decide to purchase a Sony Reader or one of the other forthcoming eink based readers that does not support the books you have already bought. What can you do other then purchase these books in a format your new device supports since the DRM has made it impossible for you to convert these books to another format.

Now if we had the books in OEBPS and a program with the new device to read in OEBPS and output what format it is we need, then what we have purchased in the past is still viable in the future. This is NOT like CD or DVD where people by choice decided to purchase the same music or movie that they had on LP, cassette, or VHS while they still had a turntable, cassette deck, or VHS recorder. This is either you stick to the old technology and can't use the new technology because your investment in the ebooks will be lost or you invest in the new technology and either do without the ebooks you have or you have to try to find them in this other format and repurchase what you can.

We don't need to lose the money we've invested in ebooks just because we want a different reading device. This is one of the major reasons why eventually, ebooks will fail and fail big time. Yes, I know I can purchase LIT format ebooks and then convert to HTML or LRF or a number of other formats, but the average person such as my mother could not do that. I could do it for her, but without me to help, she'd be buying a book that was only readable on the device she had at the time. Let's say this device breaks and is not worth getting fixed and the device she gets next won't support her ebooks, then her investment is lost. The other major issue with the DRM is lets say she gets another device that is compatible with the format of the books she once bought, but she goes to load the books she has on this new device and while they may load, the DRM will say "this is not the same device, I won't let you view your ebooks". Now what is she to do? She'd have to go back to every online shop she got the books from and download them again. Then have to deal with the fact that there are now two copies on the computer of each book.

Basically, while this isn't too much of an issue for us geeks, it is an issue to the regular people. Come on ebook industry, stop jerking us around and get your heads out of your asses and give us something we can actually use. I have a Sony Reader now. But I cannot say that 5 years down the road, I won't find another reader that suits my needs at that time better. If I was to have purchased all my books from the Sony Connect store and the new reader doesn't support them, then all my money is gone forever.

Why is it that most new readers are coming with new formats for ebooks? Sony with BBeB (LRF) and the V2/V9 with this Wolf format. We don't need new formats. We need to standardize or make an open convertible format. Ebooks are going to die and our readers will be just one more piece of junk we'll end up tossing out as they become obsolete.

We need to figure out a way to let the industry know that we won't stand for the shot they give us. Remove the DRM so at least converters could be written. Don't jerk us abount like we were your dick. Treat us like human beings. Remember when the copy protection on Lotus 1-2-3 caused such a problem that enough people complained so Lotus dropped the copy protection. We need to do the same so the first thing they do is drop the DRM. The DRM does us no good. All it does is make screw us over but good.

Jon

[Robert Marquard]

This is why i only have books without DRM.

[mikecook]

Thanks Robert, this is good information to know.

It also looks like the v2.0 framework is close to being approved so now could be a good time to start learning about the OEBPS format.


Now, Jon...

Originally Posted by JSWolf

This is NOT like CD or DVD where people by choice decided to purchase the same music or movie that they had on LP, cassette, or VHS while they still had a turntable, cassette deck, or VHS recorder. This is either you stick to the old technology and can't use the new technology because your investment in the ebooks will be lost or you invest in the new technology and either do without the ebooks you have or you have to try to find them in this other format and repurchase what you can.

Sorry but I've heard this argument before and it still makes no sense! If your turntable/CD player/iPod (i.e. ebook reader) is old technology and you wish to go out and purchase the all-new-fangled device then yes, you DO have to go out and purchase all your favourite music (eBooks) again. This is the way the world works.

But I do agree with most of what you're saying -- When you buy a CD, you expect (and know) it will play in any manufacturers CD player. The same thing should apply with eBooks. Maybe OEBPS is taking us a step close.