Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Readers > Sony Reader > Sony Reader Dev Corner

Notices

Reply
 
Thread Tools Search this Thread
Old 10-31-2006, 11:07 AM   #1
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
reverse engineering the Reader USB protocol

Hi all,

I've reverse engineered the Sony Reader USB protocol, so that I can use my Reader in Linux. I've attached a Python program that implements
the ls, cp, rm, mkdir, touch and df commands. These present a sufficient interface to use the Reader in Linux for non Connect store ebooks.

Requirements:
1) Python >= 2.5
2) libusb >= 0.1.12
2) PyUSB >= 0.3.5

To enable access for non-root users, add the following to /etc/udev/rules.d/90-prs500.rules

Code:
BUS=="usb", SYSFS{idProduct}=="029b", SYSFS{idVendor}=="054c", MODE="660", GROUP="plugdev"
and run
Code:
udevstart
or
Code:
/etc/init.d/udev restart
or whatever technique your distro uses to restart udev. If you dont know, just reboot. OSX users dont need to do this.

You may have to adjust the GROUP and the location of the rules file to suit your distribution.

Installation:
Untar, then as root run
Code:
python2.5 setup.py install
Usage:
The script
Code:
prs500.py
presents a command-line interface to the reader. Run it for usage instructions. Be careful while copying files to the reader or deleting files from the Reader, you could damage it.

If you're interested in the reader USB protocol it is documented via the api docs in the docs directory. Also giving the --log-packets option to prs500.py will produce a trace of all packets from/to the reader.

Enjoy :-),

Kovid.

EDIT: Edited from the original to give up-to-date information
Attached Files
File Type: tar libprs500-0.2.1.tar (3.58 MB, 1244 views)

Last edited by kovidgoyal; 11-14-2006 at 06:15 PM. Reason: Clarifying some instructions
kovidgoyal is offline   Reply With Quote
Old 10-31-2006, 11:10 AM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Is it possible to add .gz and .bz2 to the list of allowed file extensions for uploads to the forums?
kovidgoyal is offline   Reply With Quote
Advert
Old 10-31-2006, 11:33 AM   #3
igorsk
Wizard
igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.
 
Posts: 3,442
Karma: 300001
Join Date: Sep 2006
Location: Belgium
Device: PRS-500/505/700, Kindle, Cybook Gen3, Words Gear
Cool, looks like I don't have to worry about the Unix side after all
igorsk is offline   Reply With Quote
Old 10-31-2006, 12:07 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Thanks. Your ebook.py script was very useful in the reverse engineering as it allowed me to isolate individual operations in the logs.
kovidgoyal is offline   Reply With Quote
Old 11-01-2006, 08:29 AM   #5
gseryakov
Enthusiast
gseryakov began at the beginning.
 
Posts: 27
Karma: 10
Join Date: Oct 2006
Device: Sony Reader
Quote:
Originally Posted by kovidgoyal
Thanks. Your ebook.py script was very useful in the reverse engineering as it allowed me to isolate individual operations in the logs.
Kovid,

AFAIU you logged somehow the USB activity and then associated groups of USB "calls" with the calls on the ebookusb level. How did you do that? Is it true the ebookusb API is merely just a reflection of operations on USB level? Do you have something like table of correspondence between USB and ebookusb? If not how that table can be produced from your python code? From your logs?

What I am trying to tell - runnable source code is great but documentations is also a very interesting thing.
gseryakov is offline   Reply With Quote
Advert
Old 11-01-2006, 12:46 PM   #6
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Oh I agree, the documentation is the next step. This was just to let people know that it can be done easily and I'm working on it.

The way I work is by packet analysis. I use USBSnoop to capture the usb traffic generated by a particular call in ebook.py, for e.g. ls /etc. I then ananlyse the traffic pattern to decipher the SONY protocol. Yes ebookusb.dll is a wrapper around USB operations. There are two types of packets sent on the USB stream control packets and data packets. igor has already outlined the structure of the data packets, so all I have to do is identify the sequence of control packets corresponding to different function calls. The packets are encoded little endian, the first 16 bytes is usually meta-information, the remainder is the actual content. The code posted above is very preliminary, it doesn't really identify the structure of most packets, it simply duplicates the recorded USB data stream. At some point next week I will post better code that documents the structure and sequence of packets.

Generating your own table of packets is very easy. Just use USBSnoop and format the logs using spike.pl. You have to understand the USB 1.1 specification though. If that's too much trouble, wait till next week when I post the more structured code.

Last edited by kovidgoyal; 11-01-2006 at 12:49 PM.
kovidgoyal is offline   Reply With Quote
Old 11-01-2006, 01:08 PM   #7
arivero
Guru
arivero knows what time it isarivero knows what time it isarivero knows what time it isarivero knows what time it isarivero knows what time it isarivero knows what time it isarivero knows what time it isarivero knows what time it isarivero knows what time it isarivero knows what time it isarivero knows what time it is
 
arivero's Avatar
 
Posts: 607
Karma: 2157
Join Date: Oct 2005
Device: NCR3125, Nokia 770,...
The people in the original lbhook project got to use the USB protocol (actually, a test module included in Sony source code, and still there in the Reader source code) to communicate between an script in the librei and another program in Unix, then launching shell, pppd and all that. Are you on that?
arivero is offline   Reply With Quote
Old 11-01-2006, 01:58 PM   #8
igorsk
Wizard
igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.
 
Posts: 3,442
Karma: 300001
Join Date: Sep 2006
Location: Belgium
Device: PRS-500/505/700, Kindle, Cybook Gen3, Words Gear
I will try and document the whole protocol one day...
igorsk is offline   Reply With Quote
Old 11-01-2006, 04:00 PM   #9
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
@arivero
No I'm not, I'm kind of new to ebook readers and SONY products in general. A link to the source code would be helpful. Thanks.

@igorsk
That would certainly make my job easier.
kovidgoyal is offline   Reply With Quote
Old 11-01-2006, 04:12 PM   #10
igorsk
Wizard
igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.
 
Posts: 3,442
Karma: 300001
Join Date: Sep 2006
Location: Belgium
Device: PRS-500/505/700, Kindle, Cybook Gen3, Words Gear
http://developer.berlios.de/project/...?group_id=3700 -> Librié USB root console
Also, check the USBTG_EBOOK4_20060801.tgz package from http://www.sony.net/Products/Linux/D.../PRS-500U.html
igorsk is offline   Reply With Quote
Old 11-01-2006, 08:14 PM   #11
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Thanks the sony sources look interesting though most of the really useful stuff appears to be in an object file.
kovidgoyal is offline   Reply With Quote
Old 11-01-2006, 09:41 PM   #12
igorsk
Wizard
igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.igorsk ought to be getting tired of karma fortunes by now.
 
Posts: 3,442
Karma: 300001
Join Date: Sep 2006
Location: Belgium
Device: PRS-500/505/700, Kindle, Cybook Gen3, Words Gear
Um, the .o files are just compiled sources... they all should be there.
Make sure you check the "test" dir.
igorsk is offline   Reply With Quote
Old 11-01-2006, 10:43 PM   #13
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
You're right, the .o file I thought was important (usbtg_core.o) was just a copy, I should've looked at the Makefile
kovidgoyal is offline   Reply With Quote
Old 11-06-2006, 11:21 PM   #14
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Updated

I've attached a new version that has support for ls -l -R and getting files from the reader. Renamed to libprs500 and re-organized into a distutils package. Documentation is in the docs directory.

To install untar and as root
Code:
python setup.py install
The command
Code:
prs500.py
provides a command line interface. Just run it for usage information. Instructions for enabling access for non-root users are in the README file.

Note to igor: The implementation of file get in ebook.py v0.3 generates a whole lot of unnecessary traffic for files under 4K

EDIT: added support for the df command in ver 0.1.1
Attached Files
File Type: tar libprs500-0.1.tar (2.82 MB, 876 views)
File Type: tar libprs500-0.1.1.tar (2.95 MB, 946 views)

Last edited by kovidgoyal; 11-07-2006 at 03:08 AM.
kovidgoyal is offline   Reply With Quote
Old 11-08-2006, 08:42 PM   #15
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Write support is here!

Added write support (rm, touch, mkdir, cp host->device)
ls -lR / time reduced from 45s to 12s
Refactored the code using decorators and descriptors, its much cleaner and more robust now
The protocol is fully documented via the api documentation of prstypes.py

Since there were a lot of changes in this version, there may be regressions.
I think the interface for using non Connect store based ebooks on Linux is complete.

Be careful when issuing write commands.
Attached Files
File Type: tar libprs500-0.2.tar (3.58 MB, 922 views)
kovidgoyal is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
KDK Reverse Engineering - For Those who Cannot wait... DairyKnight Kindle Developer's Corner 0 05-14-2010 12:29 AM
Reverse-engineering the .IMP format nrapallo IMP 23 02-12-2009 01:44 PM
Reverse engineering the Cybook hard-/software srml Gen3 Developer's Corner 8 07-07-2008 04:27 PM
PC sync protocol dumky Which one should I buy? 2 01-04-2008 01:33 PM
Introduction to Reverse Engineering Software Colin Dunstan Deals and Resources (No Self-Promotion or Affiliate Links) 0 05-25-2004 11:31 AM


All times are GMT -4. The time now is 06:15 PM.


MobileRead.com is a privately owned, operated and funded community.