Project Gutenberg going Mobi?

[kovidgoyal]

Originally Posted by Peter Sorotokin

Kovid,

while I understand at to some extent share your position on uniqueness of the identifier, the standard clearly requires a globally unique id:

There are a lot of silly things in the standards, but if we all start to ignore them, the world entropy is only going to increase...

I certainly agree with the principle of adhering to standards as much as possible, that is why calibre will generate a UUID when converting from the command line. The thing is how to interpret "globally" in "globally unique". To me, "globally" cannot mean over all documents and all time, since that is fundamentally impossible. Which is why I believe the standard states that reading systems should tolerate duplicate ids.

Still, since to me the concept of having a unique id is silly on the face of it, I don't really care what is put into the package identifier, so I am quite willing to change the behavior of the calibre GUI to use a UUID as a package identifier.

[Jellby]

Originally Posted by kovidgoyal

Still, since to me the concept of having a unique id is silly on the face of it, I don't really care what is put into the package identifier, so I am quite willing to change the behavior of the calibre GUI to use a UUID as a package identifier.

Would it be enough to just append or prepend some customizable string? Like, for instance, the user name, and have "Jellby:1", "Jellby:2", ..., "Jellby:427" ...

I use a similar approach with my hand-made ePUBs, but with the date of creation instead of a number.

[=X=]

Originally Posted by kovidgoyal

The quality of the EPUB that calibre produces depends on the quality of the input you give it. There is nothing "bloated" or "slow" about EPUB files created by calibre. Basically calibre's philosophy is to make as few changes to your input HTML as possible, unlike say feedbooks, which insist on allowing only a very small and well defined subset of features in your input HTML.

And I think that is a good philosophy, but I've noticed ePUBs created with calibre take a long time to open. Not just mine it seems every ePUB I've downloaded in MobileRead (that is created with calibre) suffers the sample lag. Unlike LRF this lag happens every time I open that book. The TOC also take a very long time to load and changing chapters do too. On the TOC those a generated by calibre so by your argument they should not have such a performance hit since have complete control over their creation, save the regular expression that is passed in, but calibre also controls that syntax.

An example is google? Some of their books are pretty complicated if you take a look at them. Granted their content is lacking due to auto generated OCR. But their books are very snappy.


=X=

[kovidgoyal]

Originally Posted by =X=

The TOC also take a very long time to load and changing chapters do too. On the TOC those a generated by calibre so by your argument they should not have such a performance hit since have complete control over their creation, save the regular expression that is passed in, but calibre also controls that syntax.

calibre allows TOC entries to point to locations within HTML files. This is so that you dont have to split up your book into one HTML file per TOC entry if you dont want to. Unfortunately, ADE for some reason, that I'm sure Peter can explain, doesn't handle TOCs with this type of entry very well. For instance, if you view the same epub file with calibre's epub viewer, there is no delay in opening the TOCs. Hopefully, this behavior will eventually be fixed in ADE.

As for chapter changing being slow, as far as I know that time depends only on the complexity of the markup/size in each chapter, which is not under calibre's control.

[Jellby]

Originally Posted by =X=

And I think that is a good philosophy, but I've noticed ePUBs created with calibre take a long time to open.

Wild guess: Are these ePUBs created from the monstrous HTML generated with Word?

[Hadrien]

Originally Posted by kovidgoyal

calibre allows TOC entries to point to locations within HTML files. This is so that you dont have to split up your book into one HTML file per TOC entry if you dont want to. Unfortunately, ADE for some reason, that I'm sure Peter can explain, doesn't handle TOCs with this type of entry very well. For instance, if you view the same epub file with calibre's epub viewer, there is no delay in opening the TOCs. Hopefully, this behavior will eventually be fixed in ADE.

As for chapter changing being slow, as far as I know that time depends only on the complexity of the markup/size in each chapter, which is not under calibre's control.

In DE if you point to XHTML locations in your TOC, DE will have to parse these individual flows, that's why it takes such a long time to process. In their "best practice guide" they recommend to avoid using such things in the root elements of your TOC.

[Hadrien]

Originally Posted by kovidgoyal

The philosophical issue of whether trying to assign a unique id to electronic documents is meaningful. On this I strongly believe it is not. A consequence of that is that I believe a statement like "producing books with non unique ids is catastrophic" is just wrong.

You're the only one having a philosophical argument here: I'm pointing out that the standard requires such a unique identifier and that reading systems will use it too.
It can be catastrophic if one day we wish to handle such things such as shared annotations as we won't be able to rely on the identifier to identify a book (which is the whole purpose of an identifier).

[HarryT]

Originally Posted by Hadrien

You're the only one having a philosophical argument here: I'm pointing out that the standard requires such a unique identifier and that reading systems will use it too.
It can be catastrophic if one day we wish to handle such things such as shared annotations as we won't be able to rely on the identifier to identify a book (which is the whole purpose of an identifier).

Speaking practically rather than philosophically, is there any software which will fail if a book does not have a unique ID?

[Hadrien]

Originally Posted by HarryT

Speaking practically rather than philosophically, is there any software which will fail if a book does not have a unique ID?

I doubt that a reading software would ever fail to open or render such a book (it wouldn't make any sense) but it could fail to associate other things with the book (bookmarks, last position, annotations).

[HarryT]

I use the same ID in all the Mobi books I create (because I copy/paste the same OPF file for each book) and so far have not encountered any software in which this causes problems. All implementations of the Mobi reader use a ".mbp" file, based on the book's filename, to store annotations, bookmarks, etc.

[tompe]

Originally Posted by HarryT

I use the same ID in all the Mobi books I create (because I copy/paste the same OPF file for each book) and so far have not encountered any software in which this causes problems. All implementations of the Mobi reader use a ".mbp" file, based on the book's filename, to store annotations, bookmarks, etc.

MobiPocket is a secret format so it is not so strange that you get restrictions in possible behaviors. The point with an open standard is that it should enable new application that does not exists today.

[HarryT]

Indeed. I was simply noting that both ePub and Mobi are based on the Dublin Core Metadata standard which mandates a unique book ID, but which, in reality, doesn't actually appear to use it for anything in particular.

[DaleDe]

Originally Posted by Hadrien

I doubt that a reading software would ever fail to open or render such a book (it wouldn't make any sense) but it could fail to associate other things with the book (bookmarks, last position, annotations).

Actually I had two eBooks on my eBookwise 1150 that had the same ID and the reader would actually pick the wrong book when I tried to read it. Luckily I had the sources and the software provided a way to generate a new ID. I haven't experienced this in ePUB but then I haven't read nearly as many eBooks in this format. I think it is better safe than sorry.

Dale

[kovidgoyal]

Originally Posted by Hadrien

In DE if you point to XHTML locations in your TOC, DE will have to parse these individual flows, that's why it takes such a long time to process. In their "best practice guide" they recommend to avoid using such things in the root elements of your TOC.

I know that, I just don't understand why.

[kovidgoyal]

Originally Posted by Hadrien

You're the only one having a philosophical argument here: I'm pointing out that the standard requires such a unique identifier and that reading systems will use it too.
It can be catastrophic if one day we wish to handle such things such as shared annotations as we won't be able to rely on the identifier to identify a book (which is the whole purpose of an identifier).

And I'm saying that the standard is being idiotic. It's not possible to have a unique ID for every ebook file ever created. And any reader system that relies on this aspect of the standard is going to fall flat on its face.