PRS-500 reverse engineering the Reader USB protocol - Page 8

scotty1024 · 01-19-2007, 01:15 PM

Quote:

Originally Posted by kovidgoyal

I sympathize...but there are a few, a very few tasks that the GUI is suited for...picking items from lists is one of them.

No joy on the GUI, PyQt4 is not a player on Mac OS X so far.

kovidgoyal · 01-19-2007, 01:17 PM

I think sip is the only other dependency you will need, provided you have qt 4.2 installed. Maybe I should add instructions for installing from source to the wiki, since it will be a few months before python2.5 packages of PyQt for commonly available.

EDIT: https://libprs500.kovidgoyal.net/wiki/PyQtInstructions

kovidgoyal · 01-19-2007, 02:43 PM

Quote:

Originally Posted by scotty1024

No joy on the GUI, PyQt4 is not a player on Mac OS X so far.

Pity. The PyQt tarball for OSX doesn't compile?

scotty1024 · 01-19-2007, 03:31 PM

Quote:

Originally Posted by kovidgoyal

Pity. The PyQt tarball for OSX doesn't compile?

Compile? You didn't say I was going to have to compile something!

My compiler time is occupied on other things right now... someone else will have to pick up this ball, sorry.

chrissam42 · 01-22-2007, 05:12 AM

Quote:

Originally Posted by kovidgoyal

OK next try: Change line 363 in communicate.py to

Code:

  data = data_type(self.handle.bulk_read(self.BULK_IN_EP, size, timeout=10000))

No change (except the addition of the timeout argument in the traceback).

It doesn't feel like a timeout; the error happens after just a second or two after the GUI appears.

kovidgoyal · 01-22-2007, 12:57 PM

OK I made some changes to the way bulk reads work, this will probably break other things, but hopefully move us past this error. Apparently the USB internals in OSX work differently from Linux.

Update to latest SVN and first try
prs500 info
prs500 df

Then try the GUI.

EDIT: The revision you check out should be >= 202

chrissam42 · 01-23-2007, 04:06 AM

Command line works fine for all commands. GUI now produces:

Code:

  File "/Users/clewis/dev/libprs500/trunk/src/libprs500/gui/main.py", line 538, in establish_connection
	self.update_availabe_space(end_session=True)
  File "/Users/clewis/dev/libprs500/trunk/src/libprs500/gui/main.py", line 552, in update_availabe_space
	space = self.dev.free_space(end_session=end_session)  
  File "/Users/clewis/dev/libprs500/trunk/src/libprs500/communicate.py", line 168, in run_session
	res = func(*args, **kwargs)
  File "/Users/clewis/dev/libprs500/trunk/src/libprs500/communicate.py", line 592, in free_space
	command_number=FreeSpaceQuery.NUMBER)[0]
  File "/Users/clewis/dev/libprs500/trunk/src/libprs500/communicate.py", line 389, in _bulk_read
	packet = bulk_read_packet(data_type=data_type, size=packet_size)
  File "/Users/clewis/dev/libprs500/trunk/src/libprs500/communicate.py", line 379, in bulk_read_packet
	data = data_type(self.handle.bulk_read(self.BULK_IN_EP, rsize))
  File "/Users/clewis/dev/libprs500/trunk/src/libprs500/prstypes.py", line 717, in __init__
	" packets must have a length of atleast 16 bytes")
libprs500.errors.PacketError: 'libprs500.prstypes.FreeSpaceAnswer packets must have a length of atleast 16 bytes

kovidgoyal · 01-23-2007, 12:34 PM

I wish I had access to a machine running OSX. Does the following python script cause an error? Also I added some extra error reporting to the gui. So update and try again.

Code:

from libprs500.communicate import PRS500Device

dev = PRS500Device()
print dev.get_device_information(end_session=False)
print dev.free_space(end_session=False)

chrissam42 · 01-24-2007, 03:43 AM

Quote:

Originally Posted by kovidgoyal

I wish I had access to a machine running OSX. Does the following python script cause an error? Also I added some extra error reporting to the gui. So update and try again.

No errors on the test script (hooray!).

GUI says:

Code:

 1 Command Type: GetUSBProtocolVersion
 00000000: 0000 0000 0100 0000 0000 0000 0400 0000	................
 00000010: 0000 0000								  ....
 --
 2 Response Type: Response
 00000000: 0010 0000 0100 0000 0000 0000 0c00 0000	................
 00000010: 0000 0000 0000 0000 0000 0000 0000 0000	................
 --
 Traceback (most recent call last):
   File "/Users/clewis/dev/Reader/libprs500/trunk/src/libprs500/gui/main.py", line 523, in establish_connection
 	info = self.dev.get_device_information(end_session=False)
   File "/Users/clewis/dev/Reader/libprs500/trunk/src/libprs500/communicate.py", line 167, in run_session
 	dev.open()
   File "/Users/clewis/dev/Reader/libprs500/trunk/src/libprs500/communicate.py", line 259, in open
 	version = self._bulk_read(24, data_type=USBProtocolVersion)[0].version
   File "/Users/clewis/dev/Reader/libprs500/trunk/src/libprs500/communicate.py", line 392, in _bulk_read
 	packet = bulk_read_packet(data_type=data_type, size=packet_size)
   File "/Users/clewis/dev/Reader/libprs500/trunk/src/libprs500/communicate.py", line 379, in bulk_read_packet
 	data = data_type(self.handle.bulk_read(self.BULK_IN_EP, rsize))
   File "/Users/clewis/dev/libprs500/trunk/src/libprs500/prstypes.py", line 717, in __init__
 PacketError: 'libprs500.prstypes.USBProtocolVersion packets must have a length of atleast 16 bytes
 Unable to connect to device. Please try unplugging and reconnecting it

Xenophon · 01-24-2007, 02:11 PM

Quote:

Originally Posted by kovidgoyal

Pity. The PyQt tarball for OSX doesn't compile?

It does compile, once you get the other issues out of the way (see my previous posts). I've been travelling and haven't been able to get back to try out the GUI. I'm psyched, however, by the progress reported here in recent days!

Xenophon

Xenophon · 01-24-2007, 11:42 PM

Quote:

Originally Posted by kovidgoyal

I wish I had access to a machine running OSX. Does the following python script cause an error? Also I added some extra error reporting to the gui. So update and try again.

Code:

from libprs500.communicate import PRS500Device

dev = PRS500Device()
print dev.get_device_information(end_session=False)
print dev.free_space(end_session=False)

The test script works fine for me too. Running the GUI gets me the following (with packet logging on):

Code:

1 Command Type: GetUSBProtocolVersion
00000000: 0000 0000 0100 0000 0000 0000 0400 0000    ................
00000010: 0000 0000                                  ....
--
2 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000    ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000    ................
--
3 Answer d->h Type: USBProtocolVersion
00000000: 0500 0100 0000 0000 0000 0000 0800 0000    ................
00000010: 3031 3030 3030 3030                        01000000
--
4 Command Type: AcknowledgeBulkRead
00000000: 0010 0000 0000 0000 0000 0000 1000 0000    ................
00000010: 0500 0100 0000 0000 0000 0000 0000 0000    ................
--
5 Response Type: Response
00000000: 0010 0000 0000 0000 0000 0000 0c00 0000    ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000    ................
--
6 Command Type: SetBulkSize
00000000: 0701 0000 0100 0000 0000 0000 0400 0000    ................
00000010: 0080 0200                                  ....
--
7 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000    ................
00000010: 0701 0000 0000 0000 0000 0000 0000 0000    ................
--
8 Command Type: UnlockDevice
00000000: 0601 0000 0100 0000 0000 0000 0400 0000    ................
00000010: 2d31 0000                                  -1..
--
9 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000    ................
00000010: 0601 0000 0000 0000 0000 0000 0000 0000    ................
--
10 Command Type: SetTime
00000000: 0401 0000 0100 0000 0000 0000 1c00 0000    ................
00000010: d4fe ffff d707 0000 0100 0000 1900 0000    ................
00000020: 0400 0000 2000 0000 2500 0000              .... ...%...
--
11 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000    ................
00000010: 0401 0000 0000 0000 0000 0000 0000 0000    ................
--
12 Command Type: DeviceInfoQuery
00000000: 0101 0000 0100 0000 0000 0000 0000 0000    ................
--
13 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 1000 0000    ................
00000010: 0101 0000 0000 0000 0400 0000 5801 0000    ............X...
--
14 Answer d->h Type: DeviceInfo
00000000: 0000 1000 0000 0000 0000 0000 5801 0000    ............X...
00000010: 536f 6e79 2052 6561 6465 7200 0000 0000    Sony Reader.....
00000020: 0000 0000 0000 0000 0000 0000 0000 0000    ................
00000030: 5052 532d 3530 302f 5500 0000 0000 0000    PRS-500/U.......
00000040: 0000 0000 0000 0000 0000 0000 0000 0000    ................
00000050: 312e 302e 3030 2e32 3130 3831 0000 0000    1.0.00.21081....
00000060: 0000 0000 0100 0000 6170 706c 6963 6174    ........applicat
00000070: 696f 6e2f 782d 6262 6562 2d62 6f6f 6b00    ion/x-bbeb-book.
00000080: 0000 0000 0000 0000 0000 0000 0000 0000    ................
00000090: 0000 0000 0000 0000 0000 0000 0000 0000    ................
000000a0: 0000 0000 0000 0000 0000 0000 0000 0000    ................
000000b0: 0000 0000 0000 0000 0000 0000 0000 0000    ................
000000c0: 0000 0000 0000 0000 0000 0000 0000 0000    ................
000000d0: 0000 0000 0000 0000 0000 0000 0000 0000    ................
000000e0: 0000 0000 0000 0000 0000 0000 0000 0000    ................
000000f0: 0000 0000 0000 0000 0000 0000 0000 0000    ................
00000100: 0000 0000 0000 0000 0000 0000 0000 0000    ................
00000110: 0000 0000 0000 0000 0000 0000 0000 0000    ................
00000120: 0000 0000 0000 0000 0000 0000 0000 0000    ................
00000130: 0000 0000 0000 0000 0000 0000 0000 0000    ................
00000140: 0000 0000 0000 0000 0000 0000 0000 0000    ................
00000150: 0000 0000 0000 0000 0000 0000 0000 0000    ................
00000160: 0000 0000 0000 0000                        ........
--
15 Command Type: AcknowledgeBulkRead
00000000: 0010 0000 0000 0000 0000 0000 1000 0000    ................
00000010: 0000 1000 0000 0000 0000 0000 0000 0000    ................
--
16 Response Type: Response
00000000: 0010 0000 0000 0000 0000 0000 1000 0000    ................
00000010: 0101 0000 0000 0000 0400 0000 5801 0000    ............X...
--
17 Command Type: FreeSpaceQuery
00000000: 0301 0000 0100 0000 0000 0000 0400 0000    ................
00000010: 0000 0000                                  ....
--
18 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000    ................
00000010: 0301 0000 0000 0000 0000 0000 0000 0000    ................
--
release_interface() takes exactly 2 arguments (1 given)
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-pac
kages/libprs500-0.3.5-py2.5.egg/libprs500/gui/main.py", line 538, in establish_c
onnection
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-pac
kages/libprs500-0.3.5-py2.5.egg/libprs500/gui/main.py", line 552, in update_avai
labe_space
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-pac
kages/libprs500-0.3.5-py2.5.egg/libprs500/communicate.py", line 168, in run_sess
ion
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-pac
kages/libprs500-0.3.5-py2.5.egg/libprs500/communicate.py", line 575, in free_spa
ce
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-pac
kages/libprs500-0.3.5-py2.5.egg/libprs500/communicate.py", line 373, in _bulk_re
ad
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-pac
kages/libprs500-0.3.5-py2.5.egg/libprs500/communicate.py", line 363, in bulk_rea
d_packet
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-pac
kages/libprs500-0.3.5-py2.5.egg/libprs500/libusb.py", line 257, in bulk_read
libprs500.libusb.Error: Could not read 24 bytes on the bulk bus. Read: 0 bytes.

Seems like it's close to being there. I look forward to having things up and running!

Xenophon

kovidgoyal · 01-25-2007, 01:22 PM

This OSX bug is fixed in 0.3.6, thanks to chrissam letting me play with his OSX machine. Unfortunately, I don't understand the fix, which doesn't give me a lot of confidence when it comes to the rest of GUI. So please backup media.xml and cache.xml and test things.

Xenophon · 01-25-2007, 09:08 PM

The GUI now appears to run without problems on my Mac OS X 10.4.8 box (a 1.44GHz G4). I can list the books on the Reader, add books to the library, and so on.

A few questions:

Do you support copying from either the device's memory or the device's card back to the Library?
If yes, can you grab the metadata from the Reader and associate it with the new library entry?
Where is the library actually stored? Does the location differ on various operating systems?

The GUI is slick, and seems to work pretty well. It's also dog-slow on my machine. I'd be happy to produce whatever info might help with that (although I suspect it's more likely to be either an immature QT port or my old hardware that is primarily to blame). And, from the "give-em-an-inch-and-they-want-a-mile" department... When can we edit metadata from .rtf files? And when will you read the metadata that's already there? I've just been through the process of editing the metadata for 500 or so rtf files, and don't look forward to doing it again...

Xenophon

P.S. Thank you VERY much for your efforts so far. It is very cool to see an open source tool like this coming along so quickly!

kovidgoyal · 01-25-2007, 11:46 PM

Thanks, I'm glad to hear that it is working.

The GUI is pretty smooth on my 1.6GHz P4 laptop in both win32 and linux, so I'm inclined to believe that it's an OSX problem. Are there any other OSX users who can comment on this?

The library is stored in a file called library.db, its location is OS dependent, though at some point I will make it configurable.

Copying things from the reader to the library and/or your hard disk is on my todo list. It will transfer metadata automatically when it does that.

RTF and PDF support are also on my todo list, see the tickets on the libprs500 website for details.

Now that the basic GUI seems to be working on all platforms, I'll start working on more features. Unfortunately, I'm somewhat swamped in my day job, so progress is going to be slow for the next couple of months.

Quote:

Originally Posted by Xenophon

The GUI now appears to run without problems on my Mac OS X 10.4.8 box (a 1.44GHz G4). I can list the books on the Reader, add books to the library, and so on.

A few questions:

Do you support copying from either the device's memory or the device's card back to the Library?
If yes, can you grab the metadata from the Reader and associate it with the new library entry?
Where is the library actually stored? Does the location differ on various operating systems?

The GUI is slick, and seems to work pretty well. It's also dog-slow on my machine. I'd be happy to produce whatever info might help with that (although I suspect it's more likely to be either an immature QT port or my old hardware that is primarily to blame). And, from the "give-em-an-inch-and-they-want-a-mile" department... When can we edit metadata from .rtf files? And when will you read the metadata that's already there? I've just been through the process of editing the metadata for 500 or so rtf files, and don't look forward to doing it again...

Xenophon

P.S. Thank you VERY much for your efforts so far. It is very cool to see an open source tool like this coming along so quickly!

chrissam42 · 01-26-2007, 09:06 PM

It works now on my MacBook (though strangely still not on my desktop; I'll clean things out and try again). Thanks Kovid!

01-19-2007, 01:17 PM	#107
kovidgoyal creator of calibre Posts: 43,866 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I think sip is the only other dependency you will need, provided you have qt 4.2 installed. Maybe I should add instructions for installing from source to the wiki, since it will be a few months before python2.5 packages of PyQt for commonly available. EDIT: https://libprs500.kovidgoyal.net/wiki/PyQtInstructions Last edited by kovidgoyal; 01-19-2007 at 02:06 PM.

01-22-2007, 12:57 PM	#111
kovidgoyal creator of calibre Posts: 43,866 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	OK I made some changes to the way bulk reads work, this will probably break other things, but hopefully move us past this error. Apparently the USB internals in OSX work differently from Linux. Update to latest SVN and first try prs500 info prs500 df Then try the GUI. EDIT: The revision you check out should be >= 202 Last edited by kovidgoyal; 01-22-2007 at 03:40 PM.

01-23-2007, 12:34 PM	#113
kovidgoyal creator of calibre Posts: 43,866 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I wish I had access to a machine running OSX. Does the following python script cause an error? Also I added some extra error reporting to the gui. So update and try again. Code: from libprs500.communicate import PRS500Device dev = PRS500Device() print dev.get_device_information(end_session=False) print dev.free_space(end_session=False)

01-25-2007, 09:08 PM	#118
Xenophon curmudgeon Posts: 1,481 Karma: 5748190 Join Date: Jun 2006 Location: Redwood City, CA USA Device: Kobo Aura HD, (ex)nook, (ex)PRS-700, (ex)PRS-500	The GUI now appears to run without problems on my Mac OS X 10.4.8 box (a 1.44GHz G4). I can list the books on the Reader, add books to the library, and so on. A few questions: Do you support copying from either the device's memory or the device's card back to the Library? If yes, can you grab the metadata from the Reader and associate it with the new library entry? Where is the library actually stored? Does the location differ on various operating systems? The GUI is slick, and seems to work pretty well. It's also dog-slow on my machine. I'd be happy to produce whatever info might help with that (although I suspect it's more likely to be either an immature QT port or my old hardware that is primarily to blame). And, from the "give-em-an-inch-and-they-want-a-mile" department... When can we edit metadata from .rtf files? And when will you read the metadata that's already there? I've just been through the process of editing the metadata for 500 or so rtf files, and don't look forward to doing it again... Xenophon P.S. Thank you VERY much for your efforts so far. It is very cool to see an open source tool like this coming along so quickly!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
KDK Reverse Engineering - For Those who Cannot wait...	DairyKnight	Kindle Developer's Corner	0	05-14-2010 12:29 AM
Reverse-engineering the .IMP format	nrapallo	IMP	23	02-12-2009 01:44 PM
Reverse engineering the Cybook hard-/software	srml	Gen3 Developer's Corner	8	07-07-2008 04:27 PM
PC sync protocol	dumky	Which one should I buy?	2	01-04-2008 01:33 PM
Introduction to Reverse Engineering Software	Colin Dunstan	Deals and Resources (No Self-Promotion or Affiliate Links)	0	05-25-2004 11:31 AM

01-25-2007, 01:22 PM	#117
kovidgoyal creator of calibre Posts: 43,866 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	This OSX bug is fixed in 0.3.6, thanks to chrissam letting me play with his OSX machine. Unfortunately, I don't understand the fix, which doesn't give me a lot of confidence when it comes to the rest of GUI. So please backup media.xml and cache.xml and test things.

01-26-2007, 09:06 PM	#120
chrissam42 Member Posts: 11 Karma: 10 Join Date: Jan 2007 Location: Japan Device: Sony Reader PRS-500, Sharp/Vodafone/Softbank 904SH	It works now on my MacBook (though strangely still not on my desktop; I'll clean things out and try again). Thanks Kovid!

Advert

Advert