Adequacy of EPUB?

[Sarmat89]

Originally Posted by phillipgessert

Aren't most of OP's semantic concerns covered by epub:type?

It is a step in the right direction, but not quite.

[phillipgessert]

It seems like your idea is that by creating a highly restrictive format (forbidding things like <div>), it will force publishers to create books "the right way"—"the right way" in your mind being heavily marked-up, style-less content that can be slotted into a general "books look like this" design shell, and one that is easily searched and indexed.

I don't think a restrictive format would accomplish this, because folks will always do things a little bit wrong. It's just the nature of being a person. The best we can hope for is to make things easier to do right than they are to do wrong—and if everyone's doing it wrong, we have to reassess what "right" means.

Personally, I would rather see semantically meaningless stuff like <div> than things that are semantically incorrect, like tagging dedications as acknowledgements or introductions as forewords, which is the sort of thing I think you'd see a lot of with the kind of tagging you're talking about. Especially considering that as it is, we already see a lot of p.normal and p.blockquote type stuff.

I also don't think that sort of exceptionally precise semantics adds much to the reading experience. People usually determine what they're reading by reading it.

I don't like the idea of taking even more control out of designers' hands either. I think perhaps you have very little love for the craft of designing a book, and it's flavoring your perspective on this. The idea that fonts, indents, dropcaps etc are "not essential" is kind of mindblowing to me, but I suppose that's a matter of taste and a difference in philosophy.

Semantics and hierarchy are important, but they are not the entirety of a design, and not all aspects of a creative work can be defined semantically.

[Toxaris]

Are you familiar with 'Tiger Tiger' from Alfred Bester? Try to do that with your restrictive semantic only schema. It cannot be done and this is just an example. To say 'just use PDF' is not an answer, but a problem in itself. What is letting you to use hard semantics in ePUB already? Do you really think readers are interested if the book is semantically correct? They don't. Most are not interested in most metadata. The writer and title is usually enough, and perhaps genre. Series is a miss in the Dublin Core, but this is not the fault of the ePUB format at all. If YOU want more metadata, you are free to add it to an ePUB. There are no real restrictions there. A publisher is also not interested in the metadata.

I just cannot imagine why you propose another format to solve issues that are not there. If almost nobody sees it as an issue, it will not fly. There were more than enough examples of that in the past, DocBook one of the more known of them.

[HarryT]

Sarmat, what is your purpose here? Do you really think that your arguments are going to persuade anyone not to use ePub? You don't like ePub: fine, you're perfectly entitled to your opinions. You aren't offering any practical alternative to it, however. Exactly what is it that you want people who are creating ebooks today to do?

[eschwartz]

Originally Posted by HarryT

Sarmat, what is your purpose here? Do you really think that your arguments are going to persuade anyone not to use ePub? You don't like ePub: fine, you're perfectly entitled to your opinions. You aren't offering any practical alternative to it, however. Exactly what is it that you want people who are creating ebooks today to do?

What should ebook creators today do?
Obviously, they should learn how to prioritize things. Clearly they should be hanging out on an internet forum griping about how EPUB doesn't work, rather than waste their time making money (and satisfied customers) using an ethically wrong format.

[Hitch]

Originally Posted by eschwartz

What should ebook creators today do?
Obviously, they should learn how to prioritize things. Clearly they should be hanging out on an internet forum griping about how EPUB doesn't work, rather than waste their time making money (and satisfied customers) using an ethically wrong format.

Here, to my mind, are the biggest issues:

1. The truth is, 99.99% of all readers couldn't give two s**ts about whether or not a paragraph, a dedication, an acknowledgement, etc., are "tagged" as such. That's utterly irrelevant to them. They couldn't care less.
2. Amazon, B&N, Kobo and Apple all have self-publishing portals. There are other self-publishing portals (Smashwords, et al) being used around the world. All of these--ALL--basically work off of Word or, in Apple's case, Pages (via iAuthor). NONE of these word-process programs have remotely decent XML export capabilities.
3. Even if they did have decent XML export capabilities, here's the bigger problem: there's ZERO support out there for the MILLIONS of self-publishing authors, who are DIY'ing their books. At Amazon and B&N, they can take a correctly-styled Word file, upload it, and kablammo! Their book is made. This is done simply by using WORD's built-in styles capabilities.
4. If you use Word's built-in STYLES correctly, you can--no matter what any piker says--produce HTML that's perfectly viable for an eBook. I've done it, to prove a point.
5. There isn't ANY viable means to make XML-based eBooks that easy for the self-publisher, unless you create an even more restrictive browser-type "build your ebook here" program/app/system.
6. And while I know, factually, that two of those exist (for XHTML, mind you), DIYERs are not going to go for that. They simply will not. I can see it now: upload your XXX file. Highlight this, and indicate from our dropdown of 500 possible semantics, what this is: acknowledgement, dedication, epigraph.... Nah, NOT. For the love of GOD, you can't even get them to use bloody HEADING styles in their WP files!!!!! Or use a STYLE for ANYTHING. And you think that this will work? Force them to mark *every* single thing--every paragraph, etc., as this or that? Oh, please, somebody get me an emoty that has crocodile tears from laughing (or crying) too hard. They will NOT DO IT, period.
7. Neither Amazon, B&N, Apple nor Kobo are going to be willing to invest and create the sort of back-ends required, or to redo the massive back-end systems that they will need, to enable this type of DIY eBook in XML creation.
8. Lastly: and I realize that this holds no water with the OP--bookmakers, like me, are not going to be willing to do this. When I pull up DocBook's old specs--or 2014 specs, as some people are still out there flogging that horse--I can see at a glance that to semantically tag EVERY BLOODY ELEMENT in a book the way this guy wants will take me far more time than our current process. No argument. The idea that it will be faster or easier is just hogwash. I say that after seeing more than 3K books go through our company, so I think I have some standing to say it.
9. That means less profit for me. I'm already (as are ALL eBook-making companies, except the scammers) riding the hairy edge on profitability. I'm certainly NOT willing to do more work, for the same amount of money, to satisfy a problem that ONE person in the known World has. That's idiotic.

And lastly: other than allowing someone to sort on some maybe-included metadata--to use the first example again, "covers created by Person X," or "books published in 2014," or whatever, I have not seen any advantage listed for this not-yet-created-format. NONE. The OP persists in saying it's BETTER, but other than saying, "using semantically correct styling is BETTER," we've seen no proof whatsoever. Not a single benefit to me has been listed, other than a largely imaginary idea that somehow this would be faster. It won't be. Anyone who thinks it MIGHT be should look here: http://www.docbook.org/tdg5/en/html/ch02.html . And while the OP can say that his idea is NOT DocBook--it IS DocBook. Or, if not exactly DocBook, it's DocBooks' kissing cousin, for all intents and purposes.

To understand the reality of the entire situation, you have to ask yourself, "why did DocBook fail?" THAT is the question you have to ask yourself. By all accounts, somewhat like LaTEX, DocBook should have been taken up in large part. DocBook was, for bookmakers, what looked like an idea solution--you create one source file, and then, using the various XSLT's, you could create multiple output formats therefrom.

So: why didn't DocBook take off like a rocket?????? Hmmm. I wonder. Could it be that all the issues, I listed above, were part of the problem? It had wide OS community support; it's been kept up. There's no reason NOT to use DocBook, to this day. In fact, the OP should use it, as by and large, it will get him what he wants--fully tagged semantically-correct eBooks. Sure, he'll be stuck outputting them into ePUB or whatever--but the tagging will still be there. He says he's an ePUB maker, so this should be easy-peasy for him, and he'll get all the semantics he wants.

He claimed that there are eBook readers out there already that support XML output, so...I would imagine that it's not hard to tweak the files to create a pure XML eBook therefrom.

This way, HE can be happy, and WE can be happy.

The reality is, (gosh, I hate to bring THAT up in such a scintillating conversation) nobody is going to invest money or time or effort in another XML format, when the first one, the first large-scale one, DocBook, died like a flounder. That's the truth. This isn't a bikeshed; the whole conversation is just...ludicrous. This would only have legs if a) we convinced the IDPF it had legs, and b) more importantly, we convinced AMAZON, APPLE, etc., it had legs. Otherwise, we may as well be...well, I won't say what that is.

[dgatwood]

Just to pick nits:

Originally Posted by Hitch

[*]Amazon, B&N, Kobo and Apple all have self-publishing portals. There are other self-publishing portals (Smashwords, et al) being used around the world. All of these--ALL--basically work off of Word or, in Apple's case, Pages (via iAuthor). NONE of these word-process programs have remotely decent XML export capabilities.

Actually, all of those word processors store all of their content in XML format at this point (at least for new Word files, and for every version of Pages ever). What I think you mean is that their XML isn't semantic, and is vastly too complicated to convert sanely to EPUB or other publishing-related XML dialects without throwing a lot of information away.

Originally Posted by Hitch

Anyone who thinks it MIGHT be should look here: http://www.docbook.org/tdg5/en/html/ch02.html . And while the OP can say that his idea is NOT DocBook--it IS DocBook. Or, if not exactly DocBook, it's DocBooks' kissing cousin, for all intents and purposes.

FYI, I author my fiction books in DocBook now. But then, I'm a programmer, and I wrote my own tools to translate that into EPUB (and PDF via LaTeX), so I'm the exception that proves the rule.

Maybe if I can fix a couple of the remaining crasher bugs in WebKit's HTML editing, I might make my XML editor available to the general public, but right now, it crashes way too often. Still, it makes writing XML a lot easier than vi.

[Hitch]

Originally Posted by dgatwood

Just to pick nits:

Actually, all of those word processors store all of their content in XML format at this point (at least for new Word files, and for every version of Pages ever). What I think you mean is that their XML isn't semantic, and is vastly too complicated to convert sanely to EPUB or other publishing-related XML dialects without throwing a lot of information away.

Well...if the "nit" is that yes, Word + Pages can export XML...sure.
They CAN. But the XML is godawful. I thought, when I said that they [the programs] didn't have "remotely decent XML export" capabilities, that my intent was clear. If not, thank you for making it so. (FWIW: I spent some time looking at exporting Word-->XML-->DocBook, and after much tinkering, tweaking, yadda, decided that it was more effort than it was worth. After all, the clients don't give two s***s if we're using that or the Magic Voodoo Insta-book-maker; they just want books that they can upload at [insert retailer name here]. Worse, most of the inquirers will just out and tell me that they don't CARE if we code them by hand; if Bob down the street is making them for half the price by slapping them into Calibre, well, that's all good, as far as they're concerned.

FYI, I author my fiction books in DocBook now. But then, I'm a programmer, and I wrote my own tools to translate that into EPUB (and PDF via LaTeX), so I'm the exception that proves the rule.

Maybe if I can fix a couple of the remaining crasher bugs in WebKit's HTML editing, I might make my XML editor available to the general public, but right now, it crashes way too often. Still, it makes writing XML a lot easier than vi.

Well, then, you're just the man to make the OP happy. Make your editor, and make a reader, and Bob's yer uncle. He'll be happy; you'll <maybe> be happy, and you can't ask for more than that.

I would say, for the average bear, you'd need a frontend like Word, etc., that does the XML lifting behind the scenes. And right THERE is the problem that I see. How do you, invisibly, make the WYSIWYG component of an XML editor, that the AVERAGE AUTHOR will not find a giant PITA? They want to type, hit enter, type some more, hit enter--you know this. So, given that most refuse to learn to use Styles (yes, agreed, it's cutting off their own noses to spite their faces), how could you get them to stop and identify the element of every paragraph? And how could you make it so they wouldn't need to do that? Pattern recognition, or by-the-page-type usage, or..? Just saying.

BTW: For what it's worth, sure--I think that if someone wants to work in Markdown, or TXT, or...that using XML actually makes sense. IF you are writing in it...not going back over it afterwords and recoding everything from <p class="dedication"> to <dedication>, etc. Just like HTML (Word), if you do the work upfront, the backend is far, far easier. As a commercial formatter, though--and I know you know this part all too well, also--as my friend in the Islands would say, brah, we never get books that have all that upfront stuff done! ;-) From my perspective, I'm the streetcleaner, sweeping along behind the authorial elephant, so I have to look at what is going to generate the least amount of sweeping for me and mine.

Hitch

[Solitaire1]

I wrote the following:

Originally Posted by Solitaire1

I'm using OpenOffice.org (OO.o) and I've been generating my ebooks as PDFs, mainly because I can know exactly what my ebook will look like on my ereader. Plus, its extremely easy to generate PDFs using OO.o. I did install OO.o EPUB generator but I wasn't happy with the results (as an example, I ran into problems with non-standard style names).

I've been considering using OO.o to generate an ebook as basic HTML (OO.o can generate it as HTML 3.2), and then use Calibre to create an EPUB from that document. OO.o seems to create fairly clean HTML and can also clean up the tagging in an HTML document so I'm hoping that it will work.

Then Hitch wrote the following in response as part of a previous post:

Originally Posted by Hitch

Urk. Well...depending on what you're doing: PDF isn't really an "eBook format." I know, I know, thousands of web companies, etc., have sold ePDF "eBooks" on the net for years, but the inability of PDF to reflow (before everyone jumps on me, yes, I know that a PDF can be made that will resize and reflow, but 99.99% don't, thanks) really makes it clunky. Um...y'know, Pablo's Tutorial on ePUB-making, in the Wiki, is pretty darned good. Why don't you give that a go? I think you might be surprised at how much you like ePUB once you've made one the right way.

I do consider PDF an ebook format but its disadvantages make it unsuitable by many people for that use. But I do think it is suitable in cases where the ebook must be formatted in a specific way, and that formatting reliability is a major reason that I've been using it.

It stems from a while ago when I tried to use RTF as an ebook format on my Sony ereader. It should have been easy since Sony ereaders support RTF as ebook format and my word processor (Jarte) used it as its native format. However, I couldn't get it to consistently display the right general typeface (not a specific typeface [like Times New Roman, Arial or Courier New] but just Serif, San Serif or Monospace) within the same ebook (some paragraphs were San Serif and others Serif even though the typeface and size of the text was the same).

As I said previously, PDF was wonderful to work with because of its consistency. I can create it on my word processor (OpenOffice.org) and it will look precisely on the same on my ereader that it looks like in my word processor. Based on what I've seen in this thread and in some of the EPUB ebooks I've purchased, it seems like that is not the case with EPUB and that (along with how complicated EPUB seems to be) is the reason I haven't made much use of it and when I have tried it I haven't been pleased with the results.

It is possible that my issues with EPUB have stemmed from the fact that I've created an ebook in another format and then used tools to convert it into an EPUB. I'm going to try using Calibre to convert some of my ODT formatted ebooks to EPUB to see if I can get better results. If that isn't successful I will take a look at the tutorial you mentioned above.

[dickloraine]

If you create a pdf specially designed for your taste and device, of course that is superior to an epub, especially if you used for example latex to typeset it. But now consider you wanted to read your ebook on you smartphone. You can't read it very good, because the font is to small and you can't change it. So now you need to make a second pdf for your smartphone. Now escalate the problem to thousands of people and devices.

[dickloraine]

Oh, and epub is not really complicated. It is just html-tags and some css. Of course, if you have complicated books, it gets more difficult, but it would be more difficult in a word processor or layout engine too. And of course there is the most common case: the source format is garbage. That's not the fault of epub though. It would be difficult to make a print book out of that too. Never done print, but I would think, there are nearly no easy cases apart from if you have a source in latex and happy with that typesetting.

[Notjohn]

Originally Posted by Solitaire1

As I said previously, PDF was wonderful to work with because of its consistency. I can create it on my word processor (OpenOffice.org) and it will look precisely on the same on my ereader that it looks like in my word processor. Based on what I've seen in this thread and in some of the EPUB ebooks I've purchased, it seems like that is not the case with EPUB and that (along with how complicated EPUB seems to be) is the reason I haven't made much use of it and when I have tried it I haven't been pleased with the results.

That has not at all been my experience with epub. Except for the difficulty in printing a copy, it's been a very happy experience, now for almost four years. (I use Sigil, and the main reason I come here is to soak up the wisdom of the Sigil gurus.) I upload the same epub to Amazon, B&N, Apple (which rejects some books for violating some of its more obscure diktats), and the lesser e-tailers. I get reasonably consistent results across all platforms (and apps, in the case of Kindle books). Same with devices, though with the expected image-sizing problems in the earliest Kindle devices.

I have on occasion made a PDF of one or another book, but nobody ever bought one. More recently I made a freebie, with a chapter from each of five different books. Now that the question arises, I'll look to see if anyone has downloaded it....

Yeah, 28 people downloaded it in November. But I don't think that freebies often translate into sales.

[HarryT]

Originally Posted by dickloraine

If you create a pdf specially designed for your taste and device, of course that is superior to an epub, especially if you used for example latex to typeset it. But now consider you wanted to read your ebook on you smartphone. You can't read it very good, because the font is to small and you can't change it. So now you need to make a second pdf for your smartphone. Now escalate the problem to thousands of people and devices.

That is of course the problem, and you've stated it very well. A PDF that looks good on my Kindle Voyage with its 6" screen is not going to be good to read on my iPad with a 9.7" screen, and even worse vice versa. PDFs are great if you're producing them for a specific device or page size, but they are a poor general solution.

[Sarmat89]

Originally Posted by Toxaris

Are you familiar with 'Tiger Tiger' from Alfred Bester? Try to do that with your restrictive semantic only schema.

If you mean text laid out in asterisk form, it is impossible to do in HTML too: there is no consistent way to create a whitespace of a certain length, due to reader limitations. On the other hand, can you create a mirrored text for every reader out there? If you cannot create an effect, you should use images or PDF.
This simply isn't a convincing argument for a format giving tangible benefits for 95% of books we have.

[Sarmat89]

Originally Posted by Hitch

I can see at a glance that to semantically tag EVERY BLOODY ELEMENT in a book the way this guy wants will take me far more time than our current process. No argument.

You already have to tag 'every bloody element in a book' to make it stand out from body text. You do the same operations to assign your personal made-up style, as the predefined semantic style. This argument is rather absurd.

And while the OP can say that his idea is NOT DocBook--it IS DocBook.

Why should we use a DocBook when we make books, not computer documentation?