10-31-2006, 11:07 AM | #1 |
creator of calibre
Posts: 44,356
Karma: 23661992
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
reverse engineering the Reader USB protocol
Hi all,
I've reverse engineered the Sony Reader USB protocol, so that I can use my Reader in Linux. I've attached a Python program that implements the ls, cp, rm, mkdir, touch and df commands. These present a sufficient interface to use the Reader in Linux for non Connect store ebooks. Requirements: 1) Python >= 2.5 2) libusb >= 0.1.12 2) PyUSB >= 0.3.5 To enable access for non-root users, add the following to /etc/udev/rules.d/90-prs500.rules Code:
BUS=="usb", SYSFS{idProduct}=="029b", SYSFS{idVendor}=="054c", MODE="660", GROUP="plugdev" Code:
udevstart Code:
/etc/init.d/udev restart You may have to adjust the GROUP and the location of the rules file to suit your distribution. Installation: Untar, then as root run Code:
python2.5 setup.py install The script Code:
prs500.py If you're interested in the reader USB protocol it is documented via the api docs in the docs directory. Also giving the --log-packets option to prs500.py will produce a trace of all packets from/to the reader. Enjoy :-), Kovid. EDIT: Edited from the original to give up-to-date information Last edited by kovidgoyal; 11-14-2006 at 06:15 PM. Reason: Clarifying some instructions |
10-31-2006, 11:10 AM | #2 |
creator of calibre
Posts: 44,356
Karma: 23661992
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Is it possible to add .gz and .bz2 to the list of allowed file extensions for uploads to the forums?
|
Advert | |
|
10-31-2006, 11:33 AM | #3 |
Wizard
Posts: 3,442
Karma: 300001
Join Date: Sep 2006
Location: Belgium
Device: PRS-500/505/700, Kindle, Cybook Gen3, Words Gear
|
Cool, looks like I don't have to worry about the Unix side after all
|
10-31-2006, 12:07 PM | #4 |
creator of calibre
Posts: 44,356
Karma: 23661992
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Thanks. Your ebook.py script was very useful in the reverse engineering as it allowed me to isolate individual operations in the logs.
|
11-01-2006, 08:29 AM | #5 | |
Enthusiast
Posts: 27
Karma: 10
Join Date: Oct 2006
Device: Sony Reader
|
Quote:
AFAIU you logged somehow the USB activity and then associated groups of USB "calls" with the calls on the ebookusb level. How did you do that? Is it true the ebookusb API is merely just a reflection of operations on USB level? Do you have something like table of correspondence between USB and ebookusb? If not how that table can be produced from your python code? From your logs? What I am trying to tell - runnable source code is great but documentations is also a very interesting thing. |
|
Advert | |
|
11-01-2006, 12:46 PM | #6 |
creator of calibre
Posts: 44,356
Karma: 23661992
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Oh I agree, the documentation is the next step. This was just to let people know that it can be done easily and I'm working on it.
The way I work is by packet analysis. I use USBSnoop to capture the usb traffic generated by a particular call in ebook.py, for e.g. ls /etc. I then ananlyse the traffic pattern to decipher the SONY protocol. Yes ebookusb.dll is a wrapper around USB operations. There are two types of packets sent on the USB stream control packets and data packets. igor has already outlined the structure of the data packets, so all I have to do is identify the sequence of control packets corresponding to different function calls. The packets are encoded little endian, the first 16 bytes is usually meta-information, the remainder is the actual content. The code posted above is very preliminary, it doesn't really identify the structure of most packets, it simply duplicates the recorded USB data stream. At some point next week I will post better code that documents the structure and sequence of packets. Generating your own table of packets is very easy. Just use USBSnoop and format the logs using spike.pl. You have to understand the USB 1.1 specification though. If that's too much trouble, wait till next week when I post the more structured code. Last edited by kovidgoyal; 11-01-2006 at 12:49 PM. |
11-01-2006, 01:08 PM | #7 |
Guru
Posts: 607
Karma: 2157
Join Date: Oct 2005
Device: NCR3125, Nokia 770,...
|
The people in the original lbhook project got to use the USB protocol (actually, a test module included in Sony source code, and still there in the Reader source code) to communicate between an script in the librei and another program in Unix, then launching shell, pppd and all that. Are you on that?
|
11-01-2006, 01:58 PM | #8 |
Wizard
Posts: 3,442
Karma: 300001
Join Date: Sep 2006
Location: Belgium
Device: PRS-500/505/700, Kindle, Cybook Gen3, Words Gear
|
I will try and document the whole protocol one day...
|
11-01-2006, 04:00 PM | #9 |
creator of calibre
Posts: 44,356
Karma: 23661992
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
@arivero
No I'm not, I'm kind of new to ebook readers and SONY products in general. A link to the source code would be helpful. Thanks. @igorsk That would certainly make my job easier. |
11-01-2006, 04:12 PM | #10 |
Wizard
Posts: 3,442
Karma: 300001
Join Date: Sep 2006
Location: Belgium
Device: PRS-500/505/700, Kindle, Cybook Gen3, Words Gear
|
http://developer.berlios.de/project/...?group_id=3700 -> Librié USB root console
Also, check the USBTG_EBOOK4_20060801.tgz package from http://www.sony.net/Products/Linux/D.../PRS-500U.html |
11-01-2006, 08:14 PM | #11 |
creator of calibre
Posts: 44,356
Karma: 23661992
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Thanks the sony sources look interesting though most of the really useful stuff appears to be in an object file.
|
11-01-2006, 09:41 PM | #12 |
Wizard
Posts: 3,442
Karma: 300001
Join Date: Sep 2006
Location: Belgium
Device: PRS-500/505/700, Kindle, Cybook Gen3, Words Gear
|
Um, the .o files are just compiled sources... they all should be there.
Make sure you check the "test" dir. |
11-01-2006, 10:43 PM | #13 |
creator of calibre
Posts: 44,356
Karma: 23661992
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
You're right, the .o file I thought was important (usbtg_core.o) was just a copy, I should've looked at the Makefile
|
11-06-2006, 11:21 PM | #14 |
creator of calibre
Posts: 44,356
Karma: 23661992
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Updated
I've attached a new version that has support for ls -l -R and getting files from the reader. Renamed to libprs500 and re-organized into a distutils package. Documentation is in the docs directory.
To install untar and as root Code:
python setup.py install Code:
prs500.py Note to igor: The implementation of file get in ebook.py v0.3 generates a whole lot of unnecessary traffic for files under 4K EDIT: added support for the df command in ver 0.1.1 Last edited by kovidgoyal; 11-07-2006 at 03:08 AM. |
11-08-2006, 08:42 PM | #15 |
creator of calibre
Posts: 44,356
Karma: 23661992
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Write support is here!
Added write support (rm, touch, mkdir, cp host->device)
ls -lR / time reduced from 45s to 12s Refactored the code using decorators and descriptors, its much cleaner and more robust now The protocol is fully documented via the api documentation of prstypes.py Since there were a lot of changes in this version, there may be regressions. I think the interface for using non Connect store based ebooks on Linux is complete. Be careful when issuing write commands. |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
KDK Reverse Engineering - For Those who Cannot wait... | DairyKnight | Kindle Developer's Corner | 0 | 05-14-2010 12:29 AM |
Reverse-engineering the .IMP format | nrapallo | IMP | 23 | 02-12-2009 01:44 PM |
Reverse engineering the Cybook hard-/software | srml | Gen3 Developer's Corner | 8 | 07-07-2008 04:27 PM |
PC sync protocol | dumky | Which one should I buy? | 2 | 01-04-2008 01:33 PM |
Introduction to Reverse Engineering Software | Colin Dunstan | Deals and Resources (No Self-Promotion or Affiliate Links) | 0 | 05-25-2004 11:31 AM |