Suggestions for the FB3 format

[rsperberg]

At FictionBook.org, I posted the following suggestions for what I would like to see in the next version of the FictionBook format specification. (I've revised it slightly to accommodate the different circumstances.) I expect this site's users might have something to say on the subject too.

You can respond here, or go to the English-language forum FictionBook.org (it's a Russian-language site), specifically to this URI:

http://www.fictionbook.org/forum/vie...?p=14918#14918

Here's the post:


In a different thread, I wrote:

I have several suggestions and requests for the next version of FictionBook as a format. What's the best way to raise issues about the format?

For instance, it would nice if an FB2 file could include a short name that the e-reader could use in its header or title bar. Here's a picture that -- by virtue of a too-long name -- illustrates what I mean:

I thought I would put down my thoughts about the FictionBook format here, with suggestions for how it might be improved in its next version.The above is a simple, specific request, for a single element. The other things listed here propose larger changes.

I am new to the FB2 format, and it may be that I have missed aspects of the format in my short education, especially regarding the design principles for files. Point me to the information I need to know where something already exists, please!

And I do understand that adding elements to the FictionBook format requires e-readers to add new rendering capacities as well as to be able to differentiate between FB2 and FB3 if there are changes to the existing elements. That, of course, is true with any application and its files, but I recognize that the file creators and the application creators (of the e-readers) are two separate sets of people and these suggestions put most of the burden on those application creators.

If I didn't think the benefits would outweigh the efforts required, I wouldn't make these suggestions. But I do, so here they are:


- The first thing I feel is missing is a structure for lists.

I know from experience publishing books directly out of XML to PDF that building a bullet-proof list framework is a major task. Still, something on the level of list support in XHTML would be useful I think. Moreover, it allows the markup to better reflect the structure of the text. Just as a poem is not really a lot of p elements, neither is a list (and of course, some list items would be more than one paragraph long).

- The second thing I think would benefit readers (the human kind) would be to extend linking.

I think FictionBook should permit hyperlinks to other books in the current library or to URLs on the web. It is an electronic book, after all, being read on a computer or handheld device with a CPU in it. Shouldn't texts in the FB format include links to resources on the web? Lots of programs can can launch the default browser, open a window and supply the URL for loading. If FictionBook permits it, an e-reader can implement the feature or choose the fallback of ignoring external links.

With a link to another book in the installed library, there will need to be a way to unambigously refer to the other book. Very possibly the current identifier can be used; perhaps instead a URI-like identifier ought to be required (I haven't looked to see what Xlink allows). Note that this doesn't require an e-reader to open two books simultaneously (can any FB2-capable e-readers do that now anyway?), but just to close the current book, open the linked-to book, and jump to the referenced anchor. And, again, e-readers have the fallback of ignoring the link if this feature isn't implemented.

- FB3 should permit Flash and/or SVG files to be called and played if the e-reader has a plugin. Audio files in MP3 and/or other specified popular formats.should also be able to be called.

Well, the same argument applies here as above -- if books are electronic, why shouldn't they take full advantage of being electronic? I don't expect e-readers to be browsers or to exceed browsers' capabilities, but can't we expect e-readers to approach browsers' capabilities? It's been TEN years since Flash came out after all.

Macromedia has long made it easy to create a Flash plugin for any kind of program. But if an e-reader can't play a Flash file then the FB specification will have to require the file creator to supply a fallback -- maybe this will be one or more images with text that is only displayed when there's no Flash capability.

With an audio file, perhaps the plugin approach would work best, in which the audio capability is internal to the e-reader. Or perhaps the e-reader would launch an external audio application, similar to the way it would launch the browser when a web link is encountered.

In the current format, images are included as base64-encoded text. I don't know if this is practical with Flash or music files. This might require the format to accommodate external files, accessing them if they are in the same directory as the FBfile, or in the same zip archive.

- Lastly, I suggest that FB3 allow outside namespaces.

I'm thinking of four specific instances -- Docbook, TEI, MathML and XHTML tables.

If, for instance, an e-reader had a plug-in that allowed it to render text marked up in the Docbook format, I would like to be able to include hunks of that Docbook title in my FictionBook file without having to change the markup.

What FB3 would have to allow is a way to identify any Docbook element and what FictionBook element it should map to for those e-readers that lacked such a plugin. So maybe the description section should have an additional, optional part called mappings, where this information could be placed. Any element not in the map would fall back to the p element for rendering or be ignored.

Similarly with TEI or any other namespaced markup vocabulary.

With MathML, the fallback might more likely be an image of an equation, and consequently, I think it would have to be mentioned explicitly in the specification.

We might be getting pretty far from FictionBook with MathML, but I do think this makes the format more flexible in a worthwhile way. Of course, it doesn't really work unless e-readers can parse and render MathML. (Since I'm not going to build an e-reader, I'm pushing this from something of a theoretical standpoint, whereas every other suggestion comes from a wish to include such features in the texts I prepare.)

I note the simple table markup in FB2. Rather than duplicate the elements of the two standard table systems (CALS and XHTML), I suggest incorporating all or part of XHTML into FB3 using namespaces, similar to the way Xlink was brought in for linking. The spec might specify only a subset of elements and attributes, or it might deprecate certain usages or complexities -- for instance, tables within tables might not be permitted,. But any way you look at it tables are really, really necessary to present certain kinds of information concisely.

Lots of web pages use tables for precisely locating objects or text on the page; to avoid this kind of formatting within the content file, perhaps a table would only be allowed as a child of a certain element (the "tableholder"), and could not contain a section element.

---

These are my initial suggestions, and they definitely pull at the FictionBook format to become something much larger than it must have been conceived.

I will cross-post this list of suggestions to the MobileRead.com FBReader forum, where further suggestions may be made. My apologies for putting such major suggestions in the English-language forum of this site. I will depend on others to move the discussion to the appropriate Russian-language forum -- and to bring back the best responses and further suggestions here, so I can see them.

Thanks,

Roger Sperberg

[Jaapjan]

Actually, it sounds like you want a page-oriented webbrowser supporting xhtml. That would have about all the features you need.

[Laurens]

I've looked at the FB format some more and I see one huge problem with it for handheld devices: the fact that everything is stored in one giant XML stream. This introduces scalability problems in any program that has to render the document under low-memory conditions. It doesn't seem the FB format was designed with handheld use in mind.

Precompiled binary formats (Plucker and iSilo, for instance) are much better suited for handheld usage.

[rsperberg]

I'm willing to concede your technological point, but I'm not sure why what you say is true.

Is it because the single XML file is kept in memory?

How does making the file "precomiled" help?

I know that in the case of the FBReader, it reads zipped versions of FB2 files, so that would make the books smaller in the e-reader than in the human readable XML version (I think. Am I wrong here?).

And it looks to me like the distinct sections of the FB2 file -- description, body, binary -- could be taken advantage of in the same way that separate files could.

As for that matter, I'm not sure FBReader keeps all of a book in memory in DOM, but instead I think it uses SAX, which doesn't require a lot of memory. Would this impact your conclusions at all?

Well, you were speaking of the format. I have learned nothing that would contradict your statement that it wasn't designed with handhelds in mind.

Still, the FBReader runs on Sharp Zaurus, Siemens Simpad and Nokia 770. And the Haali Reader runs on PocketPC, Win CE 2.11, 3.0 and .NET, and WM2003. While I have experienced low-memory conditions on my Nokia 770, they have always been occasioned by Opera; I haven't seen that come up with FBReader.

I appreciate your taking the trouble to explain these issues to me.

Thanks,

Roger

[rsperberg]

Originally Posted by Jaapjan

Actually, it sounds like you want a page-oriented webbrowser supporting xhtml. That would have about all the features you need.

You're right that using browsers has made me want some of their features in an e-reader.

Browsers can handle tables too, and not many e-readers can, for that matter.

But I expect annotation and bookmarks as well as pages in an e-reader, and those aren't really browser strengths.

I guess the big difference is that browsers are html-oriented, and the reason for using namespaces would be in order to access xml vocabularies. You get much more with a well-marked-up document, an xml document, than you do with an html document. [1]

For that matter, I've explored using different browsers as the core of an e-reader, and of creating a plugin that would provide e-book-like behavior within a browser. You're perceptive to have picked up on this tendency of mine. No one else has remarked upon it, though probably my most stated opinion is that browsers have accustomed us to linking outside a document, to color, sound, motion graphics and interactivity, and it's time for e-books to keep pace with browsers and add similar capabilities.

(In my own mind, I don't imagine browsers and e-readers merging. But surely one program could handle both kinds of content. I guess it's like Gates seeing that Internet Explorer could really perform Windows Explorer's functions. Why not?)

---

[1] What I work with in my day job are semantic technologies, and specifically with topic maps. An e-book with texts in xml will adapt to what we want to do in the coming years -- in how we relate content in one document to content in other documents -- in ways that will be harder with html and webpages. (Not discounting that rdf will be brought to bear there.) So in the short term your observation is on the money, but not in the long term.

[Jaapjan]

Well, I certainly did pick up on it yes. But probably only because I work as a software developer and because of that I am bound to analyse what someone requests and how that translates to particular functionality.

As for well made markup. Perhaps you did notice my use of -x-html and not html. Well formedness is an extremely important thing for me too. Perhaps I should have said strict xhtml 1.1 then

I am not sure if I completely agree with your need to create external links, simply because much of the reference material I visit on the internet has links to other documents...which all too often no longer exist. I prefer links only within the volume set of the book(s) which would be packaged in one file.

[rsperberg]

Your point about external links is well taken.

Still I could imagine some texts intended for a school research project that might have a short-enough shelf-life that they would skirt this issue.

Or I could envision linking to a Wikipedia article or some other long-lived piece, like a newspaper article. But, as you say, the risk of demise is large enough that you'd have to be really selective about those.

And, sorry, I did miss the x reference in xhtml. I didn't realize (never looked closely at it, either) that the XHTML 1.1 recommendation allowed namespaces

[Jaapjan]

Then we are in fair agreement about what you want. Mainly an xhtml reader/browser but with pagination ability.

As for links, well, maybe I am making it bigger then it is. Should you read on a device with connectivity, it might be reasonably easy to detect documents no longer available.

I do however have a nagging suspicion that most servers you'd link to would not provide a correct HTML return code indicating 'moved' or 'no longer available'. Mmm. So maybe my point stands after all.