PRS-500 Sony Reader GPL sources download link

[scotty1024]

On reverse engineering the Librie.

I got mine when I learned that BBeB supported non-DRM content. I quickly converted the hodge podge of tools from C, C++ and C# to Java so I could run them on my Mac. The day the unit arrived I had patched the Java tools to produce my first working BBeB on a Memory Stick in 1 hour.

I grabbed a Doctorow CCL'd novel (Someone Comes to Town...) and formatted it into BBeB with my Java publishing tool and submitted it back to him to add to his online catalog of formats. He appreciated the irony of his book being available on a platform that was so heavily DRM'd it became a symbol of how badly DRM can be abused by publishers (and thus lead to a failure of the platform.)

For me reverse engineering the Librie has been more about how to produce BBeB content than about how to re-build the OS. It was nice to get english but I was doing OK managing the unit in Japanese. What would have made the unit useless was no content.

Now Sony has finally taken my order and I await its arrival, several Baen novels formatted and ready. Will I have to tweak the catalog file format on the Memory Stick from that used by the Librie? The excitement of discovery is palpable for me.

[NatCh]

Originally Posted by scotty1024

For me reverse engineering the Librie has been more about how to produce BBeB content ....

Does that mean you have some tools/utilities/apps you might could share?

[arivero]

Originally Posted by arivero

the adventure of reverse engineering the librie can be followed in a yahoo mailing list. First surprising thing: no japanese involved!!! It seems japanese hackers were not interested on the posibilities of the machine.

First step become when people dissected the file format of the upgrades. The upgrade is via the USB connection, not need to be online, using a PC application provided by sony. A person -who has not got a librie!!- did a small program able to repackage a linux filesystem and to recalculate checksums. Then some others started the "librie translation project", using this tool to create a japanese librie.

At some time in the project, they decided to introduce a trick to execute scripts: a new entry in the rc.d inits looking for a file named "hook.sh" in the memory stick, and executing it if available.

Now the source code enters play. One of the tarballs genuine from sony was a driver module for communication via the USB. And they did use of it to login. Genuine modification of a module in the source code!

(will follow later)

It follows.

After getting comunication with the machine, another guy took a look to anothed patch of the kernel source: the module to control the framebuffered e-ink. Using the knowledge *and headers* of this module, he wrote a C program to flash 4 bit images in the framebuffer independently of the sony private, not X11 and not released Display Manager. Also he did another communication module to get the images via USB from a host computer.

Then, I took these ideas and compiled the typical netpbm graphical utilities and some djvu extractors, to show how the previous upload trick could be used to write some stand alone viewers. I engineered the thing in a way that the main demo was done from bash, so no need of compiled to do another demos. I got stuck with the keyboard access, getting only some primitive reading of the keyboard

Also I compiled xvfb so that a x11 aplication can run in the librie in a very retorted way: you run it over the virtual framebuffer and a paralell process every second captures the virtual screen and uses the pnmtools to process it into a 4 bit image that can be injected into the display.

Then after some months of inactivity another bored -and brilliant- engineer took seriously the question of the keyboard, tracked the sony code and got the way to control the keyboard buffer. So very recently the first independent browser for the librie has been released. C code here, no bash tricks! Actually, Jennings his SimpleReader is not so simple, as it can use unicode.

Almost at the same time, someone took the job of getting a ppp bridge to work, old stile, over the already stablished usb terminal entry. So now it is possible also to wget from the sony, but no demo of a browser has been released.

Note that the last two advances are only some weeks old, and consider how much time the device have been in the (japanese) market!

[Slava]

>Does that mean you have some tools/utilities/apps you might could share?

Nathan, you can check it here, there are bunch of tools, including Java version by scotty1024

[NatCh]

Cool, thanks for the link, Slava!

[scotty1024]

Those are the old tools, you can expect a new set of tools once I get a chance to validate them against the Reader... in the last two days I suspect Sony tripled the installed base of BBeB capable viewing devices in the US, and that was just by shipping out review units.

Last time I checked you could still grab my BBeB formatted version off Doctorow's web site.

Now that the Reader does PDF natively I'm thinking HTMLDOC (http://www.htmldoc.org) will be useful for formating Baen ebooks (http://www.baen.com) for the Reader. But Baen content formatted with my tool will re-flow when you press the magnify button.

[igorsk]

Well, supposedly there is some kind of "reflowable" PDF which was discussed when the news of Reader's PDF support came in. Though I'm not sure whether Reader actually supports it after all, and how to make such PDFs.

[geekraver]

It would be great to at least be able to run executables on the reader (perhaps from the SD card); I for one would like to have a fbreader and djvu viewer on the device.

[arivero]

Originally Posted by geekraver

It would be great to at least be able to run executables on the reader (perhaps from the SD card); I for one would like to have a fbreader and djvu viewer on the device.

If you get some way to install in the device ("minor" GPL violation here: you should be able to reinstall), I think the librie tricks would work here.

[igorsk]

Maybe someone should write Sony and ask how to install our own kernel... who knows, maybe they will tell?

[arivero]

Originally Posted by igorsk

Maybe someone should write Sony and ask how to install our own kernel... who knows, maybe they will tell?

Actually, if they keep with the file format used for the librie, then we already have the tool. Of course we will not know until the first update from Sony.

[scotty1024]

In my experience "re-flowable PDF's" have been a sad joke. Even the BBeB re-flow capability could use some tweaks.

BBeB can be used as a page layout, in a manner similar to PDF. But it also supports auto-layout, you can feed it large blocks of text and it will layout the "pages" itself. Of course on the Librie what humans thought was a "page" and what the Librie thought was a "page" were different. Basically all the text in that large block was a "page" to the Librie. So your book could have 33 "pages" and actually be over 1000 page flips to read.

These large blocks of text limit the number of "short pages" you get when the Librie stops formatting at the end of one block and moves to the next "page". Large blocks also allow the LZW compression to produce a smaller ebook file.

If you use BBeB to do page layout, in a manner similar to PDF, you get one "page" per "page" but you lose some of the re-flow when you press the magnify button. You also get a bloated ebook file. When magnified you get short little pages stuck in between full pages, a bit annoying to read.

This was one of the questions I submitted for Sony to answer: any improvements to the BBeB format coming in the Sony Reader to address these issues on the Librie? No answer was given. If they had something to crow about I suspect they would have been crowing.

Another issue is a real italic font. In BBeB text blocks there is a pair of directives you can drop into the stream of text that pop you in and out of "italic mode". But the Librie used "fake italic", it just sheared the roman font. Looks like italic to a layman but is an abomination to a typographer.

Also, re-flowable text blocks support super/sub script, italic on/off and a horizontal escapement. So far that's all anyone has been able to tease out. There isn't even bold support.

So long as you like your text in one font, with quasi-italic, and super/sub scripts you get excellent re-flow via the magnify button. Otherwise things start getting strange as it tries to shift around page layout directives with its tiny CPU that is trying to get back to sleep mode as rapidly as it can to try and get 7500 page turns per charge.

[Slava]

Bill McCoy mentioned in his blog, that Sony Reader wont support PDF reflow.

The version of mobile/device PDF shipping with the Sony Reader 1.0 does not utilize advanced "Tagged PDF" structure information, which the desktop Adobe Reader uses to implement accessibility and reflow features. So, no, PDFs on this device are fixed-format. Admittedly, fixed-format pages are not a great match to a 6-inch display, especially when many PDFs are essentially created as print masters. Indeed this is arguably the primary reason that Sony had to support BBeB: although it has structure capabilites, PDF doesn't really solve the problem of efficiently and reliably representing flowable content that doesn't necessarily even have a single canonical paginated representation.

[negorv]

Stupid questions - just trying to save some time:
(1) Does anybody succeeded to build these sources from Sony?
(2) Do I need Montavista developer tools to do so or I can do it right under Redhat Linux (./configure arm), after installing these rpms from source download page?
(3) Does it worth to try porting QT embedded to Sony PRS-500, and is it any chance it will work with framebuffer driver supplied?

[arivero]

Originally Posted by negorv

Stupid questions - just trying to save some time:
(1) Does anybody succeeded to build these sources from Sony?
(2) Do I need Montavista developer tools to do so or I can do it right under Redhat Linux (./configure arm), after installing these rpms from source download page?
(3) Does it worth to try porting QT embedded to Sony PRS-500, and is it any chance it will work with framebuffer driver supplied?

About 3, I still think that any port of a X11 app should run the application under xvfb and copy the screen captures into the PRS-500 framebuffer driver.