Plucker pdb vs other pdb - Page 2

Jaapjan · 02-17-2006, 02:33 AM

Haha. Do not worry, I have you firmly fixed in a specific category in my mind. An in all honest, I think OpenReader is a bit of a deadend. It has not made significant progress in a long time.
At any rate, I understand you wish to move to a new XML format that supersedes all the old formats and is much more capable then the rest?

rsperberg · 02-17-2006, 07:30 AM

The point of xml is that it's extensible — if you need another element for your content, you add it. And it's self-documenting. All we need an e-reader to do is display it.

And the point of e-books, it seems to me, is that they're electronic. We should expect them to remain books but still do things print books can't do.

So in those regards you are right — but whether that means a new format or just a format and e-reader that permit extensibility and allow electronicity, I couldn't say.

Jaapjan · 02-17-2006, 10:08 AM

Well, I would say you ought to decide what direction things have to move before starting to peddle. IE: can modifications to an existing format do what you want...or do you need a new format. Incidentally, I still think xhmtl is well suited with a few minor modifications in rendering methods.

rsperberg · 02-17-2006, 05:41 PM

That is a good point.

On the other hand, I think every format so far is abysmal — for someone.

And as for XHTML, you're not traveling very far from the roost to choose that. Most every e-book format out there is derived from it or from pre-XML HTML. It's a vocabulary primarily intended for delivering text and yet it isn't rich enough.

And the rich vocabularies — Docbook, TEI — are stuffed to the gills and yet are unsuitable for general use, or even use for people outside the small domains of technical publishing and academia, respectively.

I don't fancy tacking a replacement vocabulary for handling text onto XHTML. And at the same time I recognize that any attempt to specify all the things that need to be marked up in any domain is doomed to failure.

So I say, let us agree on a framework and a small core set of elements and focus on establishing clear requirements for how content should be delivered to the reading system, and what information about the content must accompany it, and on the other hand what exactly the reading system is responsible for.

And this is for books, not the myriad of other media that also must wrestle with the issue of electronicity.

Will OpenReader do a better job of setting us up for the future? I can't say I like the way it's started if that's the task at hand. I think it stands a better chance than Sony, eReader, Mobipocket, Microsoft, IDPF, Plucker, et al., do from where they are positioned.

Jaapjan · 02-20-2006, 05:00 AM

Mayhap, mayhap. To be fair, is there a need to travel far from the roost. I suppose we both agree that XML is very extensible. Especially HTML. I agree it is not rich enough, yes. Mainly because they are only capable of representing text in the same way is paper books, which is your point all the time, yes?

So here is what I think you ask/say.

* Find a way to deliver -rich- books. Moving images, sound and pictures and complicated markup.
* Find a good manner to deliver this content to the reading device.

That brings in the following extra demands:

* Docbook and such failed because they are -complicated-. The web succeeded with HTML because it is relatively easy.
* The delivery manner must be easily implemented. If you ask me, a simply container format wil do. And the more standard, the better. Again though, difficulty means resistance. It ought be easy.

Am I correct in these two assumptions?

Laurens · 02-20-2006, 05:40 AM

The most important requirement is that the format must have minimal runtime footprint, so as to run under memory-constrained devices. Putting everything in one giant XML stream as FB2 does is not the way to go here. The XML needs to be uncompressed and parsed into an intermediate binary format for the reader to have acceptable rendering performance. In practical terms, this immediately doubles the storage requirements. That alone is a reason not to go with XML. Also, the solution might work reasonably well for fiction-length text, but it scales extremely poorly to anything bigger. Suppose you'd want to access to access the last page of a huge document like an offline Wikipedia. This would mean you'd have to parse all the preceding hundreds or thousands of pages. Without some form of binary index to facilitate random access, this would become extremely slow. (And then you end up with a binary format.)

IMO, a better solution is to have a common XML-based format that can be converted on the desktop to a precompiled binary format optimized for a particular platform with regard to its file structure (PalmOS can only store databases in PDB format in RAM), endianness (PalmOS 68K is big-endian, while ARM-based devices are little-endian) and character encoding (Windows CE supports 16-bit Unicode, while PalmOS only does 8-bit encodings).

Jaapjan · 02-20-2006, 08:19 AM

True. From a programming point of view, XML is not entirely pleasant to parse. However, contrary to binary format, it -is- extremely easy to create and...I am not entirely in agreement with you. I did not say the stream could not be compressed onto the device. There are very memory constrained compression routines that work great on text. And storage space increases all the time on devices.

What you are suggesting last sounds remarkably much like Sunrise actually, Laurens. That takes HTML and converts it to binary format for reading on PocketPC or palm. But then again, because the program is yours I suppose it fits your vision. Logically.

Laurens · 02-20-2006, 09:22 AM

Quote:

Originally Posted by Jaapjan

True. From a programming point of view, XML is not entirely pleasant to parse. However, contrary to binary format, it -is- extremely easy to create and...I am not entirely in agreement with you. I did not say the stream could not be compressed onto the device. There are very memory constrained compression routines that work great on text. And storage space increases all the time on devices.

First off, my post was not exactly a response to you, but more a general comment. (The FB2 format that Roger mentions is a single XML stream that can, optionally, be stored in a ZIP file.)

My point is that a data format has to be geared towards the environment in which it is going to be used. While it's true that decompression routines like ZLIB work fine even under low-memory conditions, random access across a compressed stream is always going to be very slow. It may have acceptable performance for fiction-length works, but it scales poorly to larger documents.

Relying on storage space increasing in future devices is wrongheaded, IMO. There are plenty of low-end devices being released even today. (The Palm Z22 for instance.) Also, in a multitasking OS, multiple processes compete for memory and programmers will always find ways to fill up that memory. (They used to say 640Kb RAM was going to be more than enough for everyone.)

Quote:

Originally Posted by Jaapjan

What you are suggesting last sounds remarkably much like Sunrise actually, Laurens. That takes HTML and converts it to binary format for reading on PocketPC or palm. But then again, because the program is yours I suppose it fits your vision. Logically.

Try opening a really large HTML file on a handheld device and notice how it slows down to a crawl. I've yet to see a handheld browser that can render the one-page version of the MySQL manual succesfully. (Pocket IE stops rendering the page about a third of the way through.) Plucker and iSilo manage to render this page completely, exactly because they optimize the document's data structures for use in memory-constrained environments and scale well to larger documents.

Jaapjan · 02-20-2006, 09:55 AM

True, but let's be fair. ZLib and many compression systems are not geared for random access compression while said binary format you mention -are- specialized, just as you say, for constrained environments.

And while you are right in saying the a data format has to be mindful of the environment it runs in, I very must -dislike- the thought of having to convert any of those files specifically for one of those devices. The idea was very much to get rid of all the different variations. At least, that is what the topic is about I think.

As for the test on the PDA, I will have to try that later. I do not have the PDA on hand here.

Laurens · 02-20-2006, 10:04 AM

Quote:

Originally Posted by Jaapjan

And while you are right in saying the a data format has to be mindful of the environment it runs in, I very must -dislike- the thought of having to convert any of those files specifically for one of those devices. The idea was very much to get rid of all the different variations.

The question is whether that is technically possible and feasible.

Remember that PalmOS devices only handle PDB format when storing databases in RAM. Unless you want to force Palm users to store documents on a memory card, you have a problem right there.

Jaapjan · 02-21-2006, 02:27 AM

This is not my personal crusade for all intent and purposes. I was merely responding to Roger.

I know Palm only supports pdb...and now we're suddenly porting pdb readers to other platforms? Seriously though, all kind of devices are locked into their own format only. I have a rocketbook 1100. Same problem as palm. Only its own format.

And I do not belief every existing device could even completely fit such a new file format. Simply because it would be -one- format and delivery method...left in the middle if this is xml or binary.

hacker · 10-14-2007, 06:39 PM

The proper location is as follows:

http://cvs.plkr.org/docs/DBFormat.html?view=co

rsperberg · 10-19-2007, 02:05 PM

Quote:

Originally Posted by hacker

The proper location is as follows:

http://cvs.plkr.org/docs/DBFormat.html?view=co

Thank you.

Roger Sperberg

ballyc27 · 04-07-2008, 05:26 PM

Hi,
post #8 stated "I think it would be nice to be able to create a Plucker file and then check it on the machine I'm using to make it."

Has there been any developments in the last 2 years that now allow this?

wallcraft · 04-07-2008, 05:53 PM

Quote:

Originally Posted by ballyc27

Has there been any developments in the last 2 years that now allow this?

FBReader reads Plucker and now runs under Windows, and under OSX (>=10.4, on Intel).

02-17-2006, 07:30 AM	#17
rsperberg Zealot Posts: 114 Karma: 10 Join Date: Jun 2005 Location: NJ Device: Kindle Voyage	The X in xml and the E in e-book The point of xml is that it's extensible — if you need another element for your content, you add it. And it's self-documenting. All we need an e-reader to do is display it. And the point of e-books, it seems to me, is that they're electronic. We should expect them to remain books but still do things print books can't do. So in those regards you are right — but whether that means a new format or just a format and e-reader that permit extensibility and allow electronicity, I couldn't say.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
What is PDB?	BookCat	Other formats	14	05-25-2016 01:28 PM
help with old .pdb ?	bwana	Reading and Management	1	03-27-2010 07:00 PM
help? pdb	Pr0tract0r	Other formats	11	01-03-2010 08:32 PM
Batch convert to plucker .pdb file.	khaoohs	Workshop	2	04-28-2006 11:23 PM
Disassembling a Plucker pdb	rsperberg	Reading and Management	0	11-21-2005 10:35 AM

02-17-2006, 02:33 AM	#16
Jaapjan Avid reader Posts: 262 Karma: 132 Join Date: Mar 2005 Location: The Netherlands Device: HTC Touch Diamond, iLiad Book Edition	Haha. Do not worry, I have you firmly fixed in a specific category in my mind. An in all honest, I think OpenReader is a bit of a deadend. It has not made significant progress in a long time. At any rate, I understand you wish to move to a new XML format that supersedes all the old formats and is much more capable then the rest?

02-17-2006, 10:08 AM	#18
Jaapjan Avid reader Posts: 262 Karma: 132 Join Date: Mar 2005 Location: The Netherlands Device: HTC Touch Diamond, iLiad Book Edition	Well, I would say you ought to decide what direction things have to move before starting to peddle. IE: can modifications to an existing format do what you want...or do you need a new format. Incidentally, I still think xhmtl is well suited with a few minor modifications in rendering methods.

02-17-2006, 05:41 PM	#19
rsperberg Zealot Posts: 114 Karma: 10 Join Date: Jun 2005 Location: NJ Device: Kindle Voyage	That is a good point. On the other hand, I think every format so far is abysmal — for someone. And as for XHTML, you're not traveling very far from the roost to choose that. Most every e-book format out there is derived from it or from pre-XML HTML. It's a vocabulary primarily intended for delivering text and yet it isn't rich enough. And the rich vocabularies — Docbook, TEI — are stuffed to the gills and yet are unsuitable for general use, or even use for people outside the small domains of technical publishing and academia, respectively. I don't fancy tacking a replacement vocabulary for handling text onto XHTML. And at the same time I recognize that any attempt to specify all the things that need to be marked up in any domain is doomed to failure. So I say, let us agree on a framework and a small core set of elements and focus on establishing clear requirements for how content should be delivered to the reading system, and what information about the content must accompany it, and on the other hand what exactly the reading system is responsible for. And this is for books, not the myriad of other media that also must wrestle with the issue of electronicity. Will OpenReader do a better job of setting us up for the future? I can't say I like the way it's started if that's the task at hand. I think it stands a better chance than Sony, eReader, Mobipocket, Microsoft, IDPF, Plucker, et al., do from where they are positioned.

02-20-2006, 05:00 AM	#20
Jaapjan Avid reader Posts: 262 Karma: 132 Join Date: Mar 2005 Location: The Netherlands Device: HTC Touch Diamond, iLiad Book Edition	Mayhap, mayhap. To be fair, is there a need to travel far from the roost. I suppose we both agree that XML is very extensible. Especially HTML. I agree it is not rich enough, yes. Mainly because they are only capable of representing text in the same way is paper books, which is your point all the time, yes? So here is what I think you ask/say. * Find a way to deliver -rich- books. Moving images, sound and pictures and complicated markup. * Find a good manner to deliver this content to the reading device. That brings in the following extra demands: * Docbook and such failed because they are -complicated-. The web succeeded with HTML because it is relatively easy. * The delivery manner must be easily implemented. If you ask me, a simply container format wil do. And the more standard, the better. Again though, difficulty means resistance. It ought be easy. Am I correct in these two assumptions?

02-20-2006, 05:40 AM	#21
Laurens Jah Blessed Posts: 1,295 Karma: 1373 Join Date: Apr 2003 Location: The Netherlands Device: iPod Touch	The most important requirement is that the format must have minimal runtime footprint, so as to run under memory-constrained devices. Putting everything in one giant XML stream as FB2 does is not the way to go here. The XML needs to be uncompressed and parsed into an intermediate binary format for the reader to have acceptable rendering performance. In practical terms, this immediately doubles the storage requirements. That alone is a reason not to go with XML. Also, the solution might work reasonably well for fiction-length text, but it scales extremely poorly to anything bigger. Suppose you'd want to access to access the last page of a huge document like an offline Wikipedia. This would mean you'd have to parse all the preceding hundreds or thousands of pages. Without some form of binary index to facilitate random access, this would become extremely slow. (And then you end up with a binary format.) IMO, a better solution is to have a common XML-based format that can be converted on the desktop to a precompiled binary format optimized for a particular platform with regard to its file structure (PalmOS can only store databases in PDB format in RAM), endianness (PalmOS 68K is big-endian, while ARM-based devices are little-endian) and character encoding (Windows CE supports 16-bit Unicode, while PalmOS only does 8-bit encodings).

02-20-2006, 08:19 AM	#22
Jaapjan Avid reader Posts: 262 Karma: 132 Join Date: Mar 2005 Location: The Netherlands Device: HTC Touch Diamond, iLiad Book Edition	True. From a programming point of view, XML is not entirely pleasant to parse. However, contrary to binary format, it -is- extremely easy to create and...I am not entirely in agreement with you. I did not say the stream could not be compressed onto the device. There are very memory constrained compression routines that work great on text. And storage space increases all the time on devices. What you are suggesting last sounds remarkably much like Sunrise actually, Laurens. That takes HTML and converts it to binary format for reading on PocketPC or palm. But then again, because the program is yours I suppose it fits your vision. Logically.

02-20-2006, 09:55 AM	#24
Jaapjan Avid reader Posts: 262 Karma: 132 Join Date: Mar 2005 Location: The Netherlands Device: HTC Touch Diamond, iLiad Book Edition	True, but let's be fair. ZLib and many compression systems are not geared for random access compression while said binary format you mention -are- specialized, just as you say, for constrained environments. And while you are right in saying the a data format has to be mindful of the environment it runs in, I very must -dislike- the thought of having to convert any of those files specifically for one of those devices. The idea was very much to get rid of all the different variations. At least, that is what the topic is about I think. As for the test on the PDA, I will have to try that later. I do not have the PDA on hand here.

02-21-2006, 02:27 AM	#26
Jaapjan Avid reader Posts: 262 Karma: 132 Join Date: Mar 2005 Location: The Netherlands Device: HTC Touch Diamond, iLiad Book Edition	This is not my personal crusade for all intent and purposes. I was merely responding to Roger. I know Palm only supports pdb...and now we're suddenly porting pdb readers to other platforms? Seriously though, all kind of devices are locked into their own format only. I have a rocketbook 1100. Same problem as palm. Only its own format. And I do not belief every existing device could even completely fit such a new file format. Simply because it would be -one- format and delivery method...left in the middle if this is xml or binary.

10-14-2007, 06:39 PM	#27
hacker Technology Mercenary Posts: 617 Karma: 2561 Join Date: Feb 2003 Location: East Lyme, CT Device: Direct Neural Implant	The proper location is as follows: http://cvs.plkr.org/docs/DBFormat.html?view=co

04-07-2008, 05:26 PM	#29
ballyc27 Member Posts: 10 Karma: 169 Join Date: Mar 2008 Device: microsoft reader	Hi, post #8 stated "I think it would be nice to be able to create a Plucker file and then check it on the machine I'm using to make it." Has there been any developments in the last 2 years that now allow this?