Hi all,

I've reverse engineered the Sony Reader USB protocol, so that I can use my Reader in Linux. I've attached a Python program that implements
the ls, cp, rm, mkdir, touch and df commands. These present a sufficient interface to use the Reader in Linux for non Connect store ebooks.

Requirements:
1) Python >= 2.5
2) libusb >= 0.1.12
2) PyUSB (http://sourceforge.net/projects/pyusb/) >= 0.3.5

To enable access for non-root users, add the following to /etc/udev/rules.d/90-prs500.rules

BUS=="usb", SYSFS{idProduct}=="029b", SYSFS{idVendor}=="054c", MODE="660", GROUP="plugdev"
and run
udevstart
or
/etc/init.d/udev restart
or whatever technique your distro uses to restart udev. If you dont know, just reboot. OSX users dont need to do this.

You may have to adjust the GROUP and the location of the rules file to suit your distribution.

Installation:
Untar, then as root run
python2.5 setup.py install

Usage:
The script
prs500.py presents a command-line interface to the reader. Run it for usage instructions. Be careful while copying files to the reader or deleting files from the Reader, you could damage it.

If you're interested in the reader USB protocol it is documented via the api docs in the docs directory. Also giving the --log-packets option to prs500.py will produce a trace of all packets from/to the reader.

Enjoy :-),

Kovid.

EDIT: Edited from the original to give up-to-date information

Is it possible to add .gz and .bz2 to the list of allowed file extensions for uploads to the forums?

Cool, looks like I don't have to worry about the Unix side after all :)

Thanks. Your ebook.py script was very useful in the reverse engineering as it allowed me to isolate individual operations in the logs.

Thanks. Your ebook.py script was very useful in the reverse engineering as it allowed me to isolate individual operations in the logs.

Kovid,

AFAIU you logged somehow the USB activity and then associated groups of USB "calls" with the calls on the ebookusb level. How did you do that? Is it true the ebookusb API is merely just a reflection of operations on USB level? Do you have something like table of correspondence between USB and ebookusb? If not how that table can be produced from your python code? From your logs?

What I am trying to tell - runnable source code is great but documentations is also a very interesting thing.

Oh I agree, the documentation is the next step. This was just to let people know that it can be done easily and I'm working on it.

The way I work is by packet analysis. I use USBSnoop to capture the usb traffic generated by a particular call in ebook.py, for e.g. ls /etc. I then ananlyse the traffic pattern to decipher the SONY protocol. Yes ebookusb.dll is a wrapper around USB operations. There are two types of packets sent on the USB stream control packets and data packets. igor has already outlined the structure of the data packets, so all I have to do is identify the sequence of control packets corresponding to different function calls. The packets are encoded little endian, the first 16 bytes is usually meta-information, the remainder is the actual content. The code posted above is very preliminary, it doesn't really identify the structure of most packets, it simply duplicates the recorded USB data stream. At some point next week I will post better code that documents the structure and sequence of packets.

Generating your own table of packets is very easy. Just use USBSnoop and format the logs using spike.pl. You have to understand the USB 1.1 specification though. If that's too much trouble, wait till next week when I post the more structured code.

The people in the original lbhook project got to use the USB protocol (actually, a test module included in Sony source code, and still there in the Reader source code) to communicate between an script in the librei and another program in Unix, then launching shell, pppd and all that. Are you on that?

I will try and document the whole protocol one day...

@arivero
No I'm not, I'm kind of new to ebook readers and SONY products in general. A link to the source code would be helpful. Thanks.

@igorsk
That would certainly make my job easier.

http://developer.berlios.de/project/showfiles.php?group_id=3700 -> Librié USB root console
Also, check the USBTG_EBOOK4_20060801.tgz package from http://www.sony.net/Products/Linux/Download/PRS-500U.html

Thanks the sony sources look interesting though most of the really useful stuff appears to be in an object file.

Um, the .o files are just compiled sources... they all should be there.
Make sure you check the "test" dir.

You're right, the .o file I thought was important (usbtg_core.o) was just a copy, I should've looked at the Makefile

I've attached a new version that has support for ls -l -R and getting files from the reader. Renamed to libprs500 and re-organized into a distutils package. Documentation is in the docs directory.

To install untar and as root
python setup.py install

The command
prs500.py
provides a command line interface. Just run it for usage information. Instructions for enabling access for non-root users are in the README file.

Note to igor: The implementation of file get in ebook.py v0.3 generates a whole lot of unnecessary traffic for files under 4K

EDIT: added support for the df command in ver 0.1.1

Added write support (rm, touch, mkdir, cp host->device)
ls -lR / time reduced from 45s to 12s
Refactored the code using decorators and descriptors, its much cleaner and more robust now
The protocol is fully documented via the api documentation of prstypes.py

Since there were a lot of changes in this version, there may be regressions.
I think the interface for using non Connect store based ebooks on Linux is complete.

Be careful when issuing write commands.

I'm on Mac OS X, which is BSD-based, so I took the plunge to see if I could use your app. I used Fink (and the FinkCommander GUI) to upgrade python 2.3 to 2.5, installed/compiled pyUSB 0.3.5, then installed libprs500-0.2. setup.py ran without errors, and I could get prs500.py to run, but it gives me errors when trying to read the device. I don't have udev on my version of Linux (it's apparently only in the Debian distros), so I had to create the udev folder, and the 90-local.rules file. That didn't affect the read errors I was getting from prs500.py, so I think udev is not related to the the problem.

I don't have python-dev or libusb-dev installed, and they aren't available as Fink packages. (Neither is udev.)

Have you tried your app on other Linux boxes?

For now, I consider libprs500-0.2 a proof-of-concept. If I could see it installed and run on other Linux boxes, then I'd call it a prototype.

(Okay, I just looked up the two terms, and they appear synonymous, so I may be splitting hairs. If I could reproduce your results, I'd call it a prototype. Perhaps others have had more success.)

Well I've tested it on Gentoo and Kubuntu. libusb has problems on Mac OSX, unfortunately I dont have a Mac on which to test it. If you look at the code in communicate.py there is a todo in there about mac osx. If you post the error messages you get that would be helpful.

In that case, it's a prototype. :)

I will post the error messages when I get a chance: my forum-reading computer is different from the one I tested libprs500 on, so I couldn't easily paste the error messages into the reply I was writing.

libusb was installable via Fink, so it may be OK now. Let me look into that, and into the todo in communicate.py--that's where the error originated.

Yeah there's no fundamental reason why this shouldn't work on OSX, besides a little tweaking of the USB parameters. I've updated the first post in the thread with the most up-to-date installation information.

I was also trying to get others to post tales of their success/failure with libprs500. Every little bit helps.

Well? Anyone??

Thanks for providing linux library, works well in Kubuntu.

The only thing that I got problem with is help message for cp command. Part of it says
"Device paths have the form: device:mountpoint/my/path"
It took me 10 min to figure out that "device" should be just the word "device" and not an actual linux device (considering there is no device created for reader it was long 10 min). Other then this the utility is very usable, thanks.

Glad to hear that it works. I'll improve the usage message for the next release.

I've isolated the problem on OSX to bulk reads failing in libusb. Unfortunately, I dont have enough access to an OSX system to debug that.

Uploaded a new version that makes the usage of cp a little clearer (I hope).

Ok, here's the errors I got from libprs500-0.2:

VeeTechV1:/sw/bin wdmoates$ prs500.py info
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/Current/bin/prs500.py", line 290, in <module>
main()
File "/Library/Frameworks/Python.framework/Versions/Current/bin/prs500.py", line 183, in main
dev.open()
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500/communicate.py", line 196, in open
version = self._bulk_read(24, data_type=USBProtocolVersion)[0].version
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500/communicate.py", line 289, in _bulk_read
packet = bulk_read_packet(data_type=data_type, size=packet_size)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500/communicate.py", line 281, in bulk_read_packet
data = data_type(self.handle.bulkRead(PRS500Device.PRS500 _BULK_IN_EP, size))
usb.USBError: usb_bulk_read: An error occured during read (see messages above)

I upgraded to libprs500-0.2.1, and here's the errors I get:

VeeTechV1:/sw/bin wdmoates$ prs500.py info
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/Current/bin/prs500.py", line 290, in <module>
main()
File "/Library/Frameworks/Python.framework/Versions/Current/bin/prs500.py", line 183, in main
dev.open()
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500/communicate.py", line 192, in open
version = self._bulk_read(24, data_type=USBProtocolVersion)[0].version
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500/communicate.py", line 286, in _bulk_read
packet = bulk_read_packet(data_type=data_type, size=packet_size)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500/communicate.py", line 278, in bulk_read_packet
data = data_type(self.handle.bulkRead(PRS500Device.PRS500 _BULK_IN_EP, size))
usb.USBError: usb_bulk_read: An error occured during read (see messages above)

Still getting the error with bulk_read_packet. This is my fault, because I didn't get the error message to you sooner.

However, I did have some other successes:

Only Python 2.3 comes with Mac OS X versions 10.3 and 10.4, and Apple's documentation on it is sketchy. I first tried using Fink to upgrade, where I had to install xfree86 to upgrade to Python 2.5. The second time, I found out Python's website had Mac installers for Python 2.5 (you can find them here (http://www.python.org/download/mac/) ), so I did that on my second Mac, which made things much easier.

In both cases, I've gotten libusb and pyUSB to install without much problem--they're mostly idiot-proof. :)

Here's the libprs500 commands which perform without error:

prs500.py
prs500.py --version
(when unplugged) prs500.py info

When I use these commands, I get the same bulk_read_packet error I mentioned above:

prs500.py info
prs500.py df
prs500.py cat
prs500.py ls

If you need more info, let me know.

Thanks, yeah that's the same error I was getting in my testing. Unfortunately, the problem seems to be in the libusb implementation on the Mac. Basically any attempt to perform bulk read/writes fails.

I don't know enough about USB internals on the Mac to debug it. Knowing me I'm probably not going to be able resist the challenge to learn though ;-)

I've been lurking on this thread for a bit hoping someone would solve the OS X problem.

But I decided to take the plunge and see if I could help work on things. However, I'm a bit of unix noob and can't seem to find python 2.5, I downloaded fink but I can only see 2.3 and 2.4.

http://www.python.org/download/

I will get around to working on OSX, it's just not high on my list of priorities. Right now I'm working on reverse engineering the LRF format and building a GUI.

Let me know if you need any help with that :)

I'd be interested, too!

Well I have written a meta information reader/writer for LRF in python, that seems to be more complete than existing tools. That's what I need for the GUI atm, so I will work on the rest of the LRF format later. If you want to track development
svn co https://svn.kovidgoyal.net/code/prs-500

To install,
python setup.py develop
Commands are: prs500, prs500-gui, lrf-meta
Additional requirements: pyxml and PyQt4

I've investigated some on the Mac end and made some progress, it appears that the usb interface is idle normally.

AppleUSBEHCI[0x327c000]::UIMCreateControlTransfer - control transaction while in idlesuspend - restarting bus

If i quickly repeat i get a different error message, with communication timing out.

File "/Library/Frameworks/Python.framework/Versions/Current/bin/prs500.py", line 290, in <module>
main()
File "/Library/Frameworks/Python.framework/Versions/Current/bin/prs500.py", line 183, in main
dev.open()
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500/communicate.py", line 190, in open
res = self._send_validated_command(GetUSBProtocolVersion ())
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500/communicate.py", line 229, in _send_validated_command
res = self._send_command(command, response_type=response_type, timeout=timeout)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500/communicate.py", line 219, in _send_command
response = response_type(self.handle.controlMsg(0xc0, 0x81, Response.SIZE, timeout=timeout))
usb.USBError: usb_control_msg(DeviceRequestTO): transaction timed out

but by repeating the command quickly I was able to get it to run... sort of. If I (for lack of a better word) hammer the command I get

Unable to find Sony Reader. Is it connected?

so, I guess this is progress?

Interesting, it may be that by adding a wake up command we can fix this. Could you sync to the latest version of libprs500, that would make it easier for me to keep track

svn co https://svn.kovidgoyal.net/code/prs-500

As root python setup.py develop to install and then the command is prs500.

I'll sync up when I get home from work.

I installed it, but it didn't seem to go right. It is throwing an error importing VERSION, I tried to bypass that, and it's throwing an error importing TerminalController.

:blink:

What did noy go well the python setup.py develop step or running prs500?

libprs500 did not seem to copy right, I deleted the egg files and ran setup.py from the tarball in the first post and it worked again.

so, I went back to that and got the USB prober working. more happens than the system logs led me to believe. first it looks for drivers, some of these it tries several times.

7862.807 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.iokit.IOUSBMassStorageClass, score: 0, wildCard = 0
7862.807 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleUSBTCKeyboard, score: 0, wildCard = 0
7862.808 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleUSBTrackpad, score: 0, wildCard = 0
7862.808 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleUSBMergeNub, score: 0, wildCard = 0
7862.809 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.iokit.IOUSBHIDDriver, score: 0, wildCard = 0
7862.809 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.iokit.IOUSBMassStorageClass, score: 0, wildCard = 0
7862.809 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleUSBCDCECMData, score: 0, wildCard = 0
7862.809 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleUSBOpticalMouse, score: 0, wildCard = 0
7862.809 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleUSBCDCWCM, score: 0, wildCard = 0
7862.811 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleUSBCDCACMData, score: 0, wildCard = 0
7862.811 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleUSBIrDA, score: 0, wildCard = 0
7862.811 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleUSBCDCDMM, score: 0, wildCard = 0
7862.811 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleIRController, score: 0, wildCard = 0
7862.811 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleWWANVerizon, score: 0, wildCard = 0
7862.811 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.iokit.IOUSBHIDDriver, score: 0, wildCard = 0
7862.811 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleUSBCDCACMControl, score: 0, wildCard = 0
7862.811 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.PreventMediaMountDriver, score: 0, wildCard = 0
7862.812 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleSM56KUSBAudio, score: 0, wildCard = 0
7862.812 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleHIDMouse, score: 0, wildCard = 0
7862.812 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleUSBDisplays, score: 0, wildCard = 0
7862.812 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleVerizonSupportKicker, score: 0, wildCard = 0
7862.813 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleUSBCDCEEM, score: 0, wildCard = 0
7862.814 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleUSBAudio, score: 0, wildCard = 0
7862.814 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.driver.AppleUSBCDCECMControl, score: 0, wildCard = 0

that was with a "sony" filter up

here's a more detailed look at what's happening

193.221 [6] Sony Portable Reader System[0x3461d00]:FindConfig (1)
193.221 [5] Sony Portable Reader System[0x3461d00]::SetConfiguration Found an interface (0x39115a9)
193.221 [5] Sony Portable Reader System[0x3461d00]::SetConfiguration Attaching an interface (0x3aa5800)
193.221 [6] IOUSBInterface[0x3aa5800]::start - opening device[0x3461d00]
193.221 [5] CompositeDriverInterestHandler received kIOMessageServiceIsAttemptingOpen with argument: 0
193.221 [6] IOUSBDevice[0x3461d00]::handleOpen - IOUSBInterface[0x3aa5800] added to open set
193.221 [5] Sony Portable Reader System[0x3461d00]::SetConfiguration matching to interfaces (0)
193.227 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.iokit.IOUSBMassStorageClass, score: 0, wildCard = 0
193.227 [6] Matched: bConfigurationValue (1) bInterfaceNumber (0)
193.227 [6] Didn't Match: idVendor (1761,1356) idProduct (55349,667)
193.227 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.iokit.IOUSBMassStorageClass, score: 0, wildCard = 0
193.227 [6] Matched: bConfigurationValue (1) bInterfaceNumber (0)
193.227 [6] Didn't Match: idVendor (2060,1356) idProduct (544,667)
193.227 [5] Finding driver for interface #0 of Sony Portable Reader System, matching personality using com.apple.iokit.IOUSBMassStorageClass, score: 0, wildCard = 0
193.227 [6] Matched: bConfigurationValue (1) bInterfaceNumber (0)
193.227 [6] Didn't Match: idVendor (65535,1356) idProduct (65535,667)

at first I was thinking this happened when your program tries to connect, but i investigated more and found that it does the same thing when the reader is plugged in. so this is likely caused by the reset that occurs.

YES!

I got it to work!

in communicate.py

I commented out line 189

self.handle.reset()

edit: for clarification, this was the source in the tarball in the first post, and on OS X v10.4.8 (this probably doesn't matter)

That's great. In fact the reset has already been removed from svn, it was left in from testing. Its good that it works on OS X.

As for libprs500 not copying right, the contents of the libprs500 directory under trunk should look like this:

libprs500:
cli
communicate.py
errors.py
gui
__init__.py
lrf
prstypes.py

libprs500/cli:
__init__.py
main.py
terminfo.py

libprs500/gui:
database.py
editbook.py
editbook.ui
images
images.qrc
images_rc.py
__init__.py
main.py
main.ui

libprs500/gui/images:
cherubs.jpg
clear.png
edit.png
fileopen.png
minus.png
mycomputer.png
plus.png
reader.png

libprs500/lrf:
__init__.py
meta.py

I believe that was all there, however when i ran "sudo python2.5 setup.py develop" it didn't associate/copy things properly, and running prs500.py wouldn't import from the libprs500.

Oh the develop command doesn't copy anything, it only creates an egg link to the source directory. The command to use is nor prs500 not prs500.py. prs500 is created by the develop command.

ahhhh

<--- noob!

Looking at the svn, it looks like you're making a gui wrapper for this, how is that going?

Ok i've tried the new code out (well downloaded last night) and it still has a reset in it causing problems, at line 258 in communicate.py. removing that info and ls seem to work, but books only gets 1 book in and vomits with


Books in main memory:
1984 by George Orwell at /Data/media/books/CBUS12510000100J.lrf
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/Current/bin/prs500", line 8, in <module>
load_entry_point('libprs500==0.2.1', 'console_scripts', 'prs500')()
File "/Users/seamus/prs-500/trunk/libprs500/cli/main.py", line 211, in main
print book
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 11: ordinal not in range(128)

Oh I apologize, I had not committed the reset change from my working copy to svn. It's in now. Also a possible fix for the books problem.

The GUI is coming along...right now it can display the list of books (with their covers) on the device. I'm working on the code to manage an sqlite based library that keeps books in various formats as well as metadata for them. After that the only hurdle will be transferring books to the reader in an intelligent fashion.

awesome, that worked well! :D

there was 1 or 2 extra .encode's in prs-500.py around line 211 (not sure if i put them there or not, i was trying something along the lines of what you did, but python must have known i was a noob) and the edit marks for r120 caused some problems, so I just got rid of them (and the old lines)

unfortunately Ot4 hasn't been aquafied, and i haven't tried to set it up in darwin yet (not sure if that will work). so i couldn't check out the hard work you've put into the gui :(

Thanks for putting all this work in!

Qt: http://www.trolltech.com/developer/downloads/qt/mac

sip: http://www.riverbankcomputing.co.uk/sip/download.php

pyqt http://www.riverbankcomputing.co.uk/pyqt/download.php


You will of course need Xcode to build them

in communicate.py

I commented out line 189

self.handle.reset()

edit: for clarification, this was the source in the tarball in the first post, and on OS X v10.4.8 (this probably doesn't matter)

seamusfp and kovidgoyal,

I looked in my version of communicate.py, found the line of code you mentioned, commented it out, and got prs500 to work, too! Glad to see someone figured out how to make it run on a Mac. (BTW, I'm on OS X v10.3.9.)

I'm going to look into Apple's documentation on installers, and see if this can be packaged in an installer. Making DMGs is easy, making stuff installable is something else.

P.S.: seamusfp, in case you didn't know, Xcode is on the OS X installation disc.

When I release a stable version, libprs500 will be installable via a single command, provided you have python2.5, Qt4, PyQt4 and libusb installed already. I'm not at all familiar with apple's packaging technologies, so I cant help you much there.

You may already know this, but I just tried to run the gui and got this error


File "/Users/seamus/prs-500/trunk/libprs500/gui/main.py", line 514, in timerEvent
self.establish_connection()
File "/Users/seamus/prs-500/trunk/libprs500/gui/main.py", line 550, in establish_connection
space = self.dev.available_space()
File "/Users/seamus/prs-500/trunk/libprs500/communicate.py", line 205, in run_session
res = func(*args, **kwargs)
File "/Users/seamus/prs-500/trunk/libprs500/communicate.py", line 495, in available_space
res = self._send_validated_command(FreeSpaceQuery(path), timeout=5000) # Timeout needs to be increased as it takes time to read card
File "/Users/seamus/prs-500/trunk/libprs500/communicate.py", line 300, in _send_validated_command
PRS500Device._validate_response(res, type=command.type, number=cnumber)
File "/Users/seamus/prs-500/trunk/libprs500/communicate.py", line 245, in _validate_response
"\nrnumber: expected="+hex(number)+" actual="+hex(res.rnumber))
libprs500.errors.ProtocolError: Inavlid response.
type: expected=0x1 actual=0xfffffffe00000000L
rnumber: expected=0x53 actual=0x53


actually as I read the code that's throwing the error, it looks like you already know this can happen.

but it looks nice!

While I know it can happen, it should not be happening. Can ypu put a print path statement above the line

res = self._send_validated_command(FreeSpaceQuery(path), timeout=5000) # Timeout needs to be increased as it takes time to read card

and try the command prs500 df

before i left for work i tried prs500 df, and it worked, so i'm pretty confused about why the gui can't do the same thing. when i get home i'll try adding the print path.

i also sometimes get a seg fault, though that may be from some of the things i was trying to get it working :) also after running the gui, the command line app doesn't work unless i disconnect and reconnect the reader, though that's probably because i haven't had a chance to let the gui end gracefully yet.

Sync up before testing as I've added a possible fix.

Ok, i've synced up. Still not working :(

though it's a new error


Python(1442,0xa000cfc0) malloc: *** error for object 0x31bdeb0: incorrect checksum for freed object - object was probably modified after being freed, break at szone_error to debug
Python(1442,0xa000cfc0) malloc: *** set a breakpoint in szone_error to debug
Bus error


also after this happens the command line app no longer works, it throws an error similar to what the gui was throwing with rev120.

edit: also if i start prs500-gui then connect the reader, the progress bar moves back and forth but nothing actually happens.

Hmm well I'm going to wait till the GUI stabilizes a bit more before I track down this bug, since I cant replicate it and guessing is getting me nowhere.

fair enough!

Good luck!

I couldn't figure out where it was coming from (the error wasn't too usefull...) I may try stepping through some of it...

The location of my svn server has changed. The new command to check out the development version is:
svn co https://svn.kovidgoyal.net/code/prs-500

The connection problems in the GUI should be fixed now. If not try increasing the timeout value in the open method in communicate.py. It's at 10000 right now.

another error that doesn't make sense :(


Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/Current/bin/prs500-gui", line 8, in <module>
load_entry_point('libprs500==0.2.1', 'gui_scripts', 'prs500-gui')()
File "/Users/seamus/prs-500/trunk/libprs500/gui/main.py", line 755, in main
gui = MainWindow(window, options.log_packets)
File "/Users/seamus/prs-500/trunk/libprs500/gui/main.py", line 593, in __init__
self.library_model.set_data(LibraryDatabase(str(se lf.database_path)))
File "/Users/seamus/prs-500/trunk/libprs500/gui/main.py", line 192, in set_data
self._data = self.db.get_table(self.FIELDS)
File "/Users/seamus/prs-500/trunk/libprs500/gui/database.py", line 77, in get_table
cur = self.con.execute("select " + cols + " from books_meta")
sqlite3.OperationalError: no such column: rating


I'm guessing it's cause I don't have sql properly installed? that's the only reason I can think of.

My fault I forgot to mention that the database format has changed since the last revision. Either delete ~/library.db or

sqlite3 ~/library.db
sqlite> alter table books_meta add column rating integer;

I tried, new messages, but they seem to be related. still in the establish connection section.

File "/Users/seamus/prs-500/trunk/libprs500/gui/main.py", line 715, in establish_connection
Python(393,0xa000cfc0) malloc: *** error for object 0x31bb1a0: incorrect checksum for freed object - object was probably modified after being freed, break at szone_error to debug
Python(393,0xa000cfc0) malloc: *** set a breakpoint in szone_error to debug
space = self.dev.available_space()
File "/Users/seamus/prs-500/trunk/libprs500/communicate.py", line 157, in run_session
if not dev.handle: dev.open()
File "/Users/seamus/prs-500/trunk/libprs500/communicate.py", line 226, in open
version = self._bulk_read(24, data_type=USBProtocolVersion)[0].version
File "/Users/seamus/prs-500/trunk/libprs500/communicate.py", line 320, in _bulk_read
packet = bulk_read_packet(data_type=data_type, size=packet_size)
File "/Users/seamus/prs-500/trunk/libprs500/communicate.py", line 312, in bulk_read_packet
data = data_type(self.handle.bulkRead(PRS500Device.PRS500 _BULK_IN_EP, size))
File "/Users/seamus/prs-500/trunk/libprs500/prstypes.py", line 625, in __init__
raise PacketError(str(self.__class__)[7:-2] + " packets must have a length of atleast 16 bytes")
PacketError: 'libprs500.prstypes.USBProtocolVersion packets must have a length of atleast 16 bytes
Unable to connect to device. Please try unplugging and reconnecting it
Unable to connect to device. Please try unplugging and reconnecting it

I did a little searching to see what's up, it looks like malloc in OSX is sort of broken,

The implementation of realloc on Darwin never frees memory, so when using a
large buffer size on incomplete reads you end up with lots of small strings that
actually take up a large allocation.

I haven't found a workaround yet...

Edit: I found someone who was getting the malloc errors in some other program and traced it to adding/removing whitespace I removed lines and that eliminated the malloc portion of the error, but the rest remains a la

File "/Users/seamus/prs-500/trunk/libprs500/gui/main.py", line 689, in establish_connection
space = self.dev.available_space()
File "/Users/seamus/prs-500/trunk/libprs500/communicate.py", line 157, in run_session
if not dev.handle: dev.open()
File "/Users/seamus/prs-500/trunk/libprs500/communicate.py", line 226, in open
version = self._bulk_read(24, data_type=USBProtocolVersion)[0].version
File "/Users/seamus/prs-500/trunk/libprs500/communicate.py", line 320, in _bulk_read
packet = bulk_read_packet(data_type=data_type, size=packet_size)
File "/Users/seamus/prs-500/trunk/libprs500/communicate.py", line 312, in bulk_read_packet
data = data_type(self.handle.bulkRead(PRS500Device.PRS500 _BULK_IN_EP, size))
File "/Users/seamus/prs-500/trunk/libprs500/prstypes.py", line 625, in __init__
raise PacketError(str(self.__class__)[7:-2] + " packets must have a length of atleast 16 bytes")
PacketError: 'libprs500.prstypes.USBProtocolVersion packets must have a length of atleast 16 bytes
Unable to connect to device. Please try unplugging and reconnecting it
Unable to connect to device. Please try unplugging and reconnecting it

Hmm can you try increasing the timeout for bulk reads. replace line 312 in communicate.py by
data = data_type(self.handle.bulkRead(PRS500Device.PRS500 _BULK_IN_EP, size, timeout=5000))

Also can you open a ticket about this at
https://libprs500.kovidgoyal.net/newticket

More convenient than a forum. Thanks

Here's a puzzle for all of you that have too much time on your hands. I just finished adding support for syncing the time to libprs500, unfortunately there's one field of 4 bytes I cant interpret. Here are the time set command packets

# 16:11 11th dec 2006
00000000: 0401 0000 0100 0000 0000 0000 1c00 0000 ................
00000010: 20fe ffff d607 0000 0c00 0000 0c00 0000 ...............
00000020: 0000 0000 0b00 0000 0800 0000 ............

# 16:20odd 11th nov 2006
00000000: 0401 0000 0100 0000 0000 0000 1c00 0000 ................
00000010: 20fe ffff d607 0000 0b00 0000 0c00 0000 ...............
00000020: 0000 0000 1500 0000 0a00 0000 ............

# 16:25odd 16th dec 2006
00000000: 0401 0000 0100 0000 0000 0000 1c00 0000 ................
00000010: 20fe ffff d607 0000 0c00 0000 1100 0000 ...............
00000020: 0000 0000 1c00 0000 2200 0000 ........"...

# 04:34odd 11th dec 2006
00000000: 0401 0000 0100 0000 0000 0000 1c00 0000 ................
00000010: 20fe ffff d607 0000 0c00 0000 0b00 0000 ...............
00000020: 0c00 0000 2200 0000 3500 0000 ...."...5...


As you can see the time information starts from byte 20 in 4byte blocks that are encoded little-endian (least significant first). Thus d6070000=2006. From byte 20 its year, month, day, hour, minute and second. The first 4 bytes are the command number, the next four are the command type and 1c is the command length. The question is what are bytes 16-19? They are the same in all time set packets I've seen (20fe ffff) and I can't figure them out? Are they and epoch, an offset of some sort, random garbage? Changing the value doesn't seem to affect the time set operation as far as I can tell, but you never know.

For those of you who actually read this far, as a reward for your patience, here's a link that documents the progress (or lack of it) I'm making with libprs500 and its GUI.
https://libprs500.kovidgoyal.net/timeline

How do you get those packets? I see no signs of the command 401 in the binaries...

oh 401 = 0x104 when decoded. hex(struct.unpack("<I", "\x04\x01\x00\x00")[0])

Ah, this makes sense then. Let me investigate a bit...

Apparently it's the timezone. 0xfffffe20 = -480, so GMT-8?

Hmm that may be right, the times themselves are reported in GMT. I've been testing the timeset operation by creating a new file and looking at its timestamp. The timezone doesn't affect that as file times are reported as epochs. Also the time entries in the media.xml file are in GMT. Can you think of some way I can test what timezone the reader is set to?

I just uploaded 0.3.0a1 to PyPI. It implements the settime command so that if you only use your reader in linux, its internal clock wont drift. Also fixed free space reporting so that it reports the correct amount of free space in main memory.

There's also a reasonably functional GUI with Drag'nDrop. Its not bug tested so if you play with it you need to backup media.xml and cache.xml to ensure you don't loose bookmarks/history.

prs500 cp prs500:/Data/database/cache/media.xml .
prs500 cp 'prs500:b:/Sony Reader/database/cache.xml' .


Installation instructions are at http://libprs500.kovidgoyal.net

If you've been using SVN you should

python setup.py develop --uninstall
svn update
python setup.py develop

After nearly two months of development, libprs500 is nearly ready to hit 0.3.0 with its shiny new GUI. All that remains is bug testing.

Screenshots, installation and usage instructions can be found at https://libprs500.kovidgoyal.net

If you succeed in installing and running libprs500 on platforms other than Linux, please let me know, preferably with installation instructions that I can add to the wiki. I have already had reports of it working on Intel based OSX and WinXP.

Please test it and if you find bugs open a ticket at
https://libprs500.kovidgoyal.net/newticket

For those of you who have installed a previous version of libprs500 and used the GUI, you will have to delete the file ~/library.db as the database format has changed. This will remove all books from the library, so if you need to preserve the database, let me know and I'll send you a script to do the trick. Hopefully, there wont be anymore database changes from now on.

Congrats to kovidgoyal on a very polished solution (judging by the screenshots). But I had some trouble installing it on WinXP. Details are given at
http://hnsws.blogspot.com/2007/01/using-reader-without-connect-software.html
But in short, the problem seems to be that the setuptools doesn't seem to recognize that libusb is installed, in spite of having copied usb.pyd into the site-packages directory. Can you help me out? Error message:
running prs500 info gave

Traceback (most recent call last):
File "L:\Python25\Scripts\prs500-script.py", line 5, in
from pkg_resources import load_entry_point
File "l:\python25\lib\site-packages\setuptools-0.6c3-py2.5.egg\pkg_resources.py", line 2479, in
File "l:\python25\lib\site-packages\setuptools-0.6c3-py2.5.egg\pkg_resources.py", line 585, in require
File "l:\python25\lib\site-packages\setuptools-0.6c3-py2.5.egg\pkg_resources.py", line 483, in resolve
pkg_resources.DistributionNotFound: pyusb>=0.3.5

Thanks.

Hmm I haven't tried it on windows yet. I'll do that and get back to you in the next couple of days.

OK various cleanups...it now runs for me in windows. I've updated the instructions in the wiki. PyXML is no longer needed and I've provided a windows installer for pyusb. Please test and let me know.

I know that some of you OS X users out there have successfully installed libprs500. I got started, but have bogged down trying to get the required libraries installed and built. Would one of you be kind enough to write a HowTo document that includes things like: Which source of libraries to use (e.g. fink, d/l and build from sourceforge, whatever)? What additional configuration steps did you need to get it all working? Any other tips and tricks?

I'd like to get this thing working (preferably WITH GUI), but I'm currently stuck.

Xenophon
(Too much thesis to write, not enough time to hack...)

Are there any libraries in particular that are giving you trouble?

Are there any libraries in particular that are giving you trouble?
I installed starting from the front of your list of basic requirements. Python 2.5 (no problem). Libusb from the Mac installer, no problem. easy_install, no problem.

Now, when I get to PyUSB, I get

sudo easy_install -U -f http://easynews.dl.sourceforge.net/sourceforge/pyusb/pyusb-0.3.5.tar.gz pyusb
Searching for pyusb
Reading http://cheeseshop.python.org/pypi/pyusb/
Couldn't retrieve index page for 'pyusb'
Scanning index of all packages (this may take a while)
Reading http://cheeseshop.python.org/pypi/
Best match: pyusb 0.3.5
Downloading http://easynews.dl.sourceforge.net/sourceforge/pyusb/pyusb-0.3.5.tar.gz
Processing pyusb-0.3.5.tar.gz
Running pyusb-0.3.5/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Tksg4F/pyusb-0.3.5/egg-dist-tmp-XQoN2_
In file included from pyusb.c:16:
pyusb.h:6:17: error: usb.h: No such file or directory
In file included from pyusb.c:16:
pyusb.h:120: error: parse error before 'usb_dev_handle'
pyusb.h:120: warning: no semicolon at end of struct or union
pyusb.h:122: error: parse error before '}' token
pyusb.h:122: warning: data definition has no type or storage class
pyusb.h:131: warning: 'struct usb_endpoint_descriptor' declared inside parameter list
pyusb.h:131: warning: its scope is only this definition or declaration, which is probably not what you want
pyusb.h:135: warning: 'struct usb_endpoint_descriptor' declared inside parameter list
pyusb.h:140: warning: 'struct usb_interface_descriptor' declared inside parameter list

and lots more errors after that.

Now what?

The automatic build wont work on OSX. Download the tarball
http://easynews.dl.sourceforge.net/sourceforge/pyusb/pyusb-0.3.5.tar.gz

Extract the files. In the pyusb directory run the commands
python2.5 setup.py build
sudo python2.5 setup.py install

If that works could you use http://cheeseshop.python.org/pypi/bdist_mpkg/
to create an mpkg that I can add to the wiki for other OSX users?

OK various cleanups...it now runs for me in windows. I've updated the instructions in the wiki. PyXML is no longer needed and I've provided a windows installer for pyusb. Please test and let me know.
Tried it. Installed pyusb with the installer, added the windows paths, restarted and again ran
easy_install prs500

This time the install did not give any errors, but said some of the scripts were already installed, and went through with the install.

Now running
prs500 info
gave a missing DLL error formsvcr80.dll
I downloaded it from http://www.dll-files.com/dllindex/dll-files.shtml?msvcr80
and put it in Windows\system32 directory, restarted, ran
prs500 info
again
Now the error message is a dialog box saying:
The procedure entry point _encoded_null could not be located in the dynamic link library msvcr80.dll
And error mesg in the cmd window:
C:\Documents and Settings\sgh>prs500 info
Traceback (most recent call last):
File "L:\Python25\scripts\prs500-script.py", line 8, in <module>
load_entry_point('libprs500==0.3.0b3', 'console_scripts', 'prs500')()
File "l:\python25\lib\site-packages\setuptools-0.6c3-py2.5.egg\pkg_resources.py", line 236, in

load_entry_point
File "l:\python25\lib\site-packages\setuptools-0.6c3-py2.5.egg\pkg_resources.py", line 2097, in

load_entry_point
File "l:\python25\lib\site-packages\setuptools-0.6c3-py2.5.egg\pkg_resources.py", line 1830, in load
File "l:\python25\lib\site-packages\libprs500-0.3.0b3-py2.5.egg\libprs500\cli\
main.py", line 25, in <module>
File "l:\python25\lib\site-packages\libprs500-0.3.0b3-py2.5.egg\libprs500\communicate.py", line 49, in

<module>
ImportError: DLL load failed: The specified procedure could not be found.

Try the attached .dll

The automatic build wont work on OSX. Download the tarball
http://easynews.dl.sourceforge.net/sourceforge/pyusb/pyusb-0.3.5.tar.gz

Extract the files. In the pyusb directory run the commands
python2.5 setup.py build
sudo python2.5 setup.py install

If that works could you use http://cheeseshop.python.org/pypi/bdist_mpkg/
to create an mpkg that I can add to the wiki for other OSX users?

That didn't fix the problem. The fix that worked happened when I took a close look at setup.py and realized that it assumes that the libusb install was done by fink (and was thus placed in /sw/...). It turns out that the latest-and-greatest libusb gets installed into /usr/local/... instead. A quick editing of setup.py to add /usr/local/include and /usr/local/lib to the extra-foo-items, and I have a successful compilation. I'll try to get an mpkg back to you soon.

Xenophon

P.S. I'm now waiting for a qt build to complete so I can try the GUI along with all the rest.

Thats great. Thanks.

I've dropped the dependency on pyusb (basically re-implemented it in pure python using ctypes). THis needs testing especially on OSX. So please


easy_install -U libprs500
prs500 info

Kovid:

I've had some success. I can now talk to the reader via the command line interface. However, I get some interesting errors from the 'books' command and the ls -lh command:

[Scipio:~] dfsuther% prs500 books
Books in main memory:
Miles, Mystery & Mayhem by Lois McMaster Bujold at /Data/media/books/Miles Mystery and Mayhem.rtf
Thraxas and the Dance of Death by Martin Scott at /Data/media/books/Thraxas_and_the_Dance_of_D.rtf
Thraxas and the Sorcerers by Martin Scott at /Data/media/books/Thraxas_and_the_Sorcerers.rtf
Brilliance of the Moon by Lian Hearn at /Data/media/books/cbuscepgi000260u.lrx
A Mankind Witch by Dave Freer at /Data/media/books/A_Mankind_Witch.rtf
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/Current/bin/prs500", line 8, in <module>
load_entry_point('libprs500==0.3.2', 'console_scripts', 'prs500')()
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.2-py2.5.egg/libprs500/cli/main.py", line 211, in main
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 11: ordinal not in range(128)
[Scipio:~] dfsuther% prs500 ls /Data/media/books
Grantville_Gazette_Volume_6.rtf March_Upcountry.rtf
Go Tell the Spartans.rtf cbuscerhi001ku04.lrx
Komarr.rtf We_Few.rtf
The_Far_Side_of_The_Stars.rtf This_Rough_Magic.rtf
1633.rtf cbuscehcl0016o08.lrx
cbuscerhi002mx0a.lrx Grantville_Gazette-Volume_VIII.rtf
Grantville_Gazette_Volume_I.rtf Miles Mystery and Mayhem.rtf
Grantville_Gazette_Volume_V.rtf Thraxas_and_the_Dance_of_D.rtf
1632.rtf Thraxas_and_the_Sorcerers.rtf
Convergent Series.rtf cbuscepgi000260u.lrx
Grantville_Gazette-Volume_IX.rtf A_Mankind_Witch.rtf
Resurgence.rtf Grantville_Gazette_Volume_IIdt.rtf
1635-The_Cannon_Law.rtf CBUS12510000200K.lrf
Transvergence.rtf Miles Errant.rtf
Thraxas_at_War (7).rtf March to the Sea.rtf
PRS500_OG.pdf cbuscerhi002f60c.lrx
Turn_The_Other_Chick.rtf Chicks n Chained Males.rtf
CBUS12510000400M.lrf 1634_-_The_Galileo_Affair.rtf
cbuscehcl0015i01.lrx Prince of Mercenaries.rtf
Thraxas.rtf 1634-The_Ram_Rebellion.rtf
High Justice.rtf King Davids Spaceship.rtf
With the Lightnings.rtf Grantville_Gazette_Volume_IV.rtf
Some_Golden_Harbor.rtf Grantville_Gazette-Volume_VII.rtf
The Shadow of the Lion.rtf cbuscerhi002o60l.lrx
Young Miles.rtf Diplomatic_Immunity.rtf
The Chicks in the Mail.rtf Thraxas_Under_Siege (8).rtf
Prince of Sparta.rtf The Mote In Gods Eye.rtf
cbuscerhi001ke0o.lrx The Gripping Hand.rtf
The Mercenary.rtf Lt. Leary, Commanding.rtf
CBUS12511000600P.lrf March to the Stars.rtf
West of Honor.rtf cbuscepgi000mo0w.lrx
The_Way_to_Glory.rtf Cordelia's Honor.rtf
cbuscepgi000cg0e.lrx Universe_v1n2.rtf
CBUS12510000300L.lrf Universe_v1n1.rtf
Death_and_Thraxas.rtf A Civil Campaign.rtf
[Scipio:~] dfsuther% prs500 ls -lh /Data/media/books
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/Current/bin/prs500", line 8, in <module>
load_entry_point('libprs500==0.3.2', 'console_scripts', 'prs500')()
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.2-py2.5.egg/libprs500/cli/main.py", line 231, in main
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.2-py2.5.egg/libprs500/cli/main.py", line 139, in ls
TypeError: object of type 'NoneType' has no len()
[Scipio:~] dfsuther%

Interesting that I can ls the contents of the directory that holds books in main memory, but I can't either ls -lh it or succeed with a prs500 books command.

Suggestions?

The ls -lh bug was a typo, fixed in svn. The books bug is basically related to character encodings and will need a little work. I've implemented a possible fix in svn. Could you checkout the svn version and try it.

Instructions are at:
https://libprs500.kovidgoyal.net/wiki/Development

Thanks.

EDIT: The possible fix has made it into version 0.3.3

I've dropped the dependency on pyusb (basically re-implemented it in pure python using ctypes). THis needs testing especially on OSX.

Thanks for all your great work on this.

Trying on OS X, "info" works but any "books" or "ls" results in:


release_interface() takes exactly 2 arguments (1 given)
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/Current/bin/prs500", line 8, in <module>
load_entry_point('libprs500==0.3.3', 'console_scripts', 'prs500')()
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.3-py2.5.egg/libprs500/cli/main.py", line 231, in main
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.3-py2.5.egg/libprs500/cli/main.py", line 128, in ls
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.3-py2.5.egg/libprs500/communicate.py", line 167, in run_session
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.3-py2.5.egg/libprs500/communicate.py", line 244, in open
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.3-py2.5.egg/libprs500/communicate.py", line 370, in _bulk_read
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.3-py2.5.egg/libprs500/communicate.py", line 360, in bulk_read_packet
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.3-py2.5.egg/libprs500/libusb.py", line 224, in bulk_read
libprs500.libusb.Error: Could not read 24 bytes on the bulk bus. Read: 0 bytes.

Hmm that's a strange error. First try unplugging your reader and reconnecting it. Then try
prs500 --log-packets ls /

Tried it again on another computer just to be sure, and got the same error. But lo and behold, when I ran it with --log-packets, it worked, and now it works fine.

If I see it again I'll try and catch it.

Here we go. "ls /" works pretty reliably now on the command line, but the gui fails consistently with the same error:


MacBook:~ clewis$ prs500-gui --log-packets
1 Command Type: GetUSBProtocolVersion
00000000: 0000 0000 0100 0000 0000 0000 0400 0000 ................
00000010: 0000 0000 ....
--
2 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
--
3 Answer d->h Type: USBProtocolVersion
00000000: 0500 0100 0000 0000 0000 0000 0800 0000 ................
00000010: 3031 3030 3030 3030 01000000
--
4 Command Type: AcknowledgeBulkRead
00000000: 0010 0000 0000 0000 0000 0000 1000 0000 ................
00000010: 0500 0100 0000 0000 0000 0000 0000 0000 ................
--
5 Response Type: Response
00000000: 0010 0000 0000 0000 0000 0000 0c00 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
--
6 Command Type: SetBulkSize
00000000: 0701 0000 0100 0000 0000 0000 0400 0000 ................
00000010: 0080 0200 ....
--
7 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 0701 0000 0000 0000 0000 0000 0000 0000 ................
--
8 Command Type: UnlockDevice
00000000: 0601 0000 0100 0000 0000 0000 0400 0000 ................
00000010: 2d31 0000 -1..
--
9 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 0601 0000 0000 0000 0000 0000 0000 0000 ................
--
10 Command Type: SetTime
00000000: 0401 0000 0100 0000 0000 0000 1c00 0000 ................
00000010: 1c02 0000 d707 0000 0100 0000 1000 0000 ................
00000020: 0d00 0000 1300 0000 2a00 0000 ........*...
--
11 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 0401 0000 0000 0000 0000 0000 0000 0000 ................
--
12 Command Type: DeviceInfoQuery
00000000: 0101 0000 0100 0000 0000 0000 0000 0000 ................
--
13 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 1000 0000 ................
00000010: 0101 0000 0000 0000 0400 0000 5801 0000 ............X...
--
14 Answer d->h Type: DeviceInfo
00000000: 0000 1000 0000 0000 0000 0000 5801 0000 ............X...
00000010: 536f 6e79 2052 6561 6465 7200 0000 0000 Sony Reader.....
00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000030: 5052 532d 3530 302f 5500 0000 0000 0000 PRS-500/U.......
00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000050: 312e 302e 3030 2e32 3130 3831 0000 0000 1.0.00.21081....
00000060: 0000 0000 0100 0000 6170 706c 6963 6174 ........applicat
00000070: 696f 6e2f 782d 6262 6562 2d62 6f6f 6b00 ion/x-bbeb-book.
00000080: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000090: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000100: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000110: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000120: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000130: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000140: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000150: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000160: 0000 0000 0000 0000 ........
--
15 Command Type: AcknowledgeBulkRead
00000000: 0010 0000 0000 0000 0000 0000 1000 0000 ................
00000010: 0000 1000 0000 0000 0000 0000 0000 0000 ................
--
16 Response Type: Response
00000000: 0010 0000 0000 0000 0000 0000 1000 0000 ................
00000010: 0101 0000 0000 0000 0400 0000 5801 0000 ............X...
--
17 Command Type: FreeSpaceQuery
00000000: 0301 0000 0100 0000 0000 0000 0400 0000 ................
00000010: 0000 0000 ....
--
18 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 0301 0000 0000 0000 0000 0000 0000 0000 ................
--
release_interface() takes exactly 2 arguments (1 given)
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.4-py2.5.egg/libprs500/gui/main.py", line 538, in establish_connection
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.4-py2.5.egg/libprs500/gui/main.py", line 552, in update_availabe_space
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.4-py2.5.egg/libprs500/communicate.py", line 168, in run_session
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.4-py2.5.egg/libprs500/communicate.py", line 571, in free_space
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.4-py2.5.egg/libprs500/communicate.py", line 369, in _bulk_read
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.4-py2.5.egg/libprs500/communicate.py", line 359, in bulk_read_packet
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.4-py2.5.egg/libprs500/libusb.py", line 224, in bulk_read
libprs500.libusb.Error: Could not read 24 bytes on the bulk bus. Read: 0 bytes.

Try the attached .dll
OK, I copied this one to the system32 directory. Incidentally, it was different from the one I had, which was 2005 date and 600 kB, this one 612 kB. But now

prs500 info

gives a runtime error dialog from python.exe

R6034 - attempt to load C runtime library incorrectly. Then the following at the prompt:

C:\Documents and Settings\sgh>prs500 info
Traceback (most recent call last):
File "L:\Python25\scripts\prs500-script.py", line 8, in <module>
load_entry_point('libprs500==0.3.0b3', 'console_scripts', 'prs500')()
File "l:\python25\lib\site-packages\setuptools-0.6c3-py2.5.egg\pkg_resources.py", line 236, in load_entry_point
File "l:\python25\lib\site-packages\setuptools-0.6c3-py2.5.egg\pkg_resources.py", line 2097, in load_entry_point
File "l:\python25\lib\site-packages\setuptools-0.6c3-py2.5.egg\pkg_resources.py", line 1830, in load
File "l:\python25\lib\site-packages\libprs500-0.3.0b3-py2.5.egg\libprs500\cli\main.py", line 25, in <module>
File "l:\python25\lib\site-packages\libprs500-0.3.0b3-py2.5.egg\libprs500\communicate.py", line 49, in <module>
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.

@chrissam: Seems to be failing at the free space query. What does prs500 df give you?

@hn88: I've dropped the pyusb dependency. Try the latest version 0.3.5.

There was a bug fix that affect windows and potentially osx users, so please upgrade to 0.3.5.

@chrissam: Seems to be failing at the free space query. What does prs500 df give you?

Upgraded to 0.3.5. Command line works every time now, but prs500-gui still fails in the same spot.

@hn88: I've dropped the pyusb dependency. Try the latest version 0.3.5.
I'll be out of station in a day or two, and have many things to tie up. Will come back in 10 days and try.

@chrissam: Hmm, I must confess I am at a loss, and since I dont have OSX, it is difficult for me to debug. But if you're willing, I have a few ideas. Checkout the svn version and in the file libprs500/gui/main.py line 523 change end_session=False to end_session=True

Instructions on checking out the svn version are at
https://libprs500.kovidgoyal.net/wiki/Development

libprs500/gui/main.py line 523 change end_session=False to end_session=True

Sadly, no change. The more detailed development error messages listed line 538 (instead of 523). I tried changing that line too to no effect.


MacBook:~/dev/libprs500/trunk clewis$ prs500-gui
release_interface() takes exactly 2 arguments (1 given)
Traceback (most recent call last):
File "/Users/clewis/dev/libprs500/trunk/src/libprs500/gui/main.py", line 538, in establish_connection
self.update_availabe_space(end_session=True)
File "/Users/clewis/dev/libprs500/trunk/src/libprs500/gui/main.py", line 552, in update_availabe_space
space = self.dev.free_space(end_session=end_session)
File "/Users/clewis/dev/libprs500/trunk/src/libprs500/communicate.py", line 168, in run_session
res = func(*args, **kwargs)
File "/Users/clewis/dev/libprs500/trunk/src/libprs500/communicate.py", line 575, in free_space
command_number=FreeSpaceQuery.NUMBER)[0]
File "/Users/clewis/dev/libprs500/trunk/src/libprs500/communicate.py", line 373, in _bulk_read
packet = bulk_read_packet(data_type=data_type, size=packet_size)
File "/Users/clewis/dev/libprs500/trunk/src/libprs500/communicate.py", line 363, in bulk_read_packet
data = data_type(self.handle.bulk_read(self.BULK_IN_EP, size))
File "/Users/clewis/dev/libprs500/trunk/src/libprs500/libusb.py", line 255, in bulk_read
'bulk bus. Read: ' + str(rsize) + ' bytes.')
libprs500.libusb.Error: Could not read 24 bytes on the bulk bus. Read: 0 bytes.

OK next try: Change line 363 in communicate.py to

data = data_type(self.handle.bulk_read(self.BULK_IN_EP, size, timeout=10000))

ibprs500 0.3.5 is working for me on Mac OS X 10.4.8 on a Macbook Pro Core 2 Duo 15".

Macbook-Pro:~/Sources/sony/libprs500/trunk scotty$ prs500 infoDevice name: Sony Reader
Device version: PRS-500/U
Software version: 1.0.00.21081
Mime type: application/x-bbeb-book
Macbook-Pro:~/Sources/sony/libprs500/trunk scotty$ prs500 ls /
Data etc mnt opt1 sbin var
bin home opt proc tmp
dev lib opt0 root usr

@scotty1024: That's good to hear. Does the GUI also work?

@scotty1024: That's good to hear. Does the GUI also work?

GUI? You know I'm a command line guy! :D

I'll let you know.

I sympathize...but there are a few, a very few tasks that the GUI is suited for...picking items from lists is one of them.

On Ubuntu edgy (with both Python 2.4 and 2.5 installed), I cannot get it to work because of dependency problems.

root@ubuntu:/disk2/sonyreader/libprs500/trunk # python2.5 setup.py develop
running develop
running egg_info
writing src/libprs500.egg-info/PKG-INFO
writing top-level names to src/libprs500.egg-info/top_level.txt
writing dependency_links to src/libprs500.egg-info/dependency_links.txt
writing entry points to src/libprs500.egg-info/entry_points.txt
writing manifest file 'src/libprs500.egg-info/SOURCES.txt'
running build_ext
Creating /usr/lib/python2.5/site-packages/libprs500.egg-link (link to src)
libprs500 0.3.5 is already the active version in easy-install.pth
Installing makelrf script to /usr/bin
Installing lrf-meta script to /usr/bin
Installing prs500 script to /usr/bin
Installing prs500-gui script to /usr/bin

Installed /mnt/hdb1/sonyreader/libprs500/trunk/src
Processing dependencies for libprs500==0.3.5
You do not have PyQt4 installed. The GUI will not work. You can obtain PyQt4 from http://www.riverbankcomputing.co.uk/pyqt/download.php

I have PyQT4 installed, but only the package that is provided by Ubuntu (python2.4-qt4). There is no python2.5-qt4, and if I was going to install PyQt4 for Python 2.5 through sources, I'd most certainly break a couple of dependency issues.

Any hints?

I installed starting from the front of your list of basic requirements. Python 2.5 (no problem). Libusb from the Mac installer, no problem. easy_install, no problem.

Now, when I get to PyUSB, I get

sudo easy_install -U -f http://easynews.dl.sourceforge.net/sourceforge/pyusb/pyusb-0.3.5.tar.gz pyusb
Searching for pyusb
Reading http://cheeseshop.python.org/pypi/pyusb/
Couldn't retrieve index page for 'pyusb'
Scanning index of all packages (this may take a while)
Reading http://cheeseshop.python.org/pypi/
Best match: pyusb 0.3.5
Downloading http://easynews.dl.sourceforge.net/sourceforge/pyusb/pyusb-0.3.5.tar.gz
Processing pyusb-0.3.5.tar.gz
Running pyusb-0.3.5/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Tksg4F/pyusb-0.3.5/egg-dist-tmp-XQoN2_
In file included from pyusb.c:16:
pyusb.h:6:17: error: usb.h: No such file or directory
In file included from pyusb.c:16:
pyusb.h:120: error: parse error before 'usb_dev_handle'
pyusb.h:120: warning: no semicolon at end of struct or union
pyusb.h:122: error: parse error before '}' token
pyusb.h:122: warning: data definition has no type or storage class
pyusb.h:131: warning: 'struct usb_endpoint_descriptor' declared inside parameter list
pyusb.h:131: warning: its scope is only this definition or declaration, which is probably not what you want
pyusb.h:135: warning: 'struct usb_endpoint_descriptor' declared inside parameter list
pyusb.h:140: warning: 'struct usb_interface_descriptor' declared inside parameter list

and lots more errors after that.

Now what?

I have the same problem when trying to install PyUSB from source (python setup.py install) on Ubuntu edge. Xenophon, you said you edited the setup script to make it work?

I have the same problem when trying to install PyUSB from source (python setup.py install) on Ubuntu edge. Xenophon, you said you edited the setup script to make it work?

Yes, but my fix may not be relevant to you. The error happens because usb.h is not being found on your include path (or perhaps it is, but not the right usb.h. On Mac OS X you might have installed libusb in either of two ways; setup.py as written supported only one of them.

More specifically... The easy way to install libusb is via the 'fink' utility (an opensource package manager for OS X). fink installs all its packages in the /sw/... tree, and that's where the current setup.py expects to find usb.h. Sadly, the version of libusb that is needed for prs500 has not yet been ported to OS X by the fink folks. So I fell back to the slightly more complicated installation method, to wit: download and install libusb from the original developers web site (out on SourceForge). That installation drops the package into the /usr/local/... tree. The only change I made to setup.py was to add /usr/local/include and /usr/local/lib in the obvious places (alongside the pre-existing /sw/include and /sw/lib that were already there).

My fix is potentially relevant to other OS X users. I will matter for you (on Ubuntu) only if you have an existing libusb install that is both (a) the version needed for prs500 and (b) installed somewhere other than the place that setup.py expects.

Xenophon

My fault, I had to install libusb-dev before running the pyusb setup! :wall:

root@ubuntu:/disk2/sonyreader/libprs500/trunk # prs500 info
Device name: Sony Reader
Device version: PRS-500/U
Software version: 1.0.00.21081
Mime type: application/x-bbeb-book

I am still not able to compile the GUI, but this is already very cool ;)

Sadly, the version of libusb that is needed for prs500 has not yet been ported to OS X by the fink folks. So I fell back to the slightly more complicated installation method, to wit: download and install libusb from the original developers web site (out on SourceForge). That installation drops the package into the /usr/local/... tree. The only change I made to setup.py was to add /usr/local/include and /usr/local/lib in the obvious places (alongside the pre-existing /sw/include and /sw/lib that were already there).

Personally I just installed libusb from the libusb macosx link on the libprs500 wiki and it worked with no changes to setup.py.

Anyway, this is all about to become much easier very soon...

Anyway, this is all about to become much easier very soon...Is that a hint, scotty1024? I'd really like for that to be a hint. :grin:

@alexander turcic: I installed PyQt4 from sources for python2.5. Didn't break any dependencies for me. Since python2.5 and python 2.4 have separate site-packages directories, I don't see why it would break any dependencies, but maybe I'm missing something.

EDIT: Also you no longer need pyusb as I re-implemented it in pure python as a part of libprs500

@kovidgoyal: unfortunately Ubuntu edge misses a lot of packages for Python 2.5... for instance, when I tried to compile PyQt4 from source, as you suggested, it would complain about "ImportError: No module named sipconfig". Alas, there is also no precompiled package available for python-sip4 for Python 2.5 - only for 2.4.

Of course I could also compile this from source, but then, who knows, what other Python 2.5 modules might be missing. I'll try it this weekend. Or I will upgrade my Ubuntu machine to feisty which has all necessary Python 2.5 packages.

I sympathize...but there are a few, a very few tasks that the GUI is suited for...picking items from lists is one of them.

No joy on the GUI, PyQt4 is not a player on Mac OS X so far.

I think sip is the only other dependency you will need, provided you have qt 4.2 installed. Maybe I should add instructions for installing from source to the wiki, since it will be a few months before python2.5 packages of PyQt for commonly available.

EDIT: https://libprs500.kovidgoyal.net/wiki/PyQtInstructions

No joy on the GUI, PyQt4 is not a player on Mac OS X so far.

Pity. The PyQt tarball for OSX doesn't compile?

Pity. The PyQt tarball for OSX doesn't compile?

Compile? You didn't say I was going to have to compile something! :D

My compiler time is occupied on other things right now... someone else will have to pick up this ball, sorry.

OK next try: Change line 363 in communicate.py to

data = data_type(self.handle.bulk_read(self.BULK_IN_EP, size, timeout=10000))


No change (except the addition of the timeout argument in the traceback).

It doesn't feel like a timeout; the error happens after just a second or two after the GUI appears.

OK I made some changes to the way bulk reads work, this will probably break other things, but hopefully move us past this error. Apparently the USB internals in OSX work differently from Linux.

Update to latest SVN and first try
prs500 info
prs500 df

Then try the GUI.

EDIT: The revision you check out should be >= 202

Command line works fine for all commands. GUI now produces:


File "/Users/clewis/dev/libprs500/trunk/src/libprs500/gui/main.py", line 538, in establish_connection
self.update_availabe_space(end_session=True)
File "/Users/clewis/dev/libprs500/trunk/src/libprs500/gui/main.py", line 552, in update_availabe_space
space = self.dev.free_space(end_session=end_session)
File "/Users/clewis/dev/libprs500/trunk/src/libprs500/communicate.py", line 168, in run_session
res = func(*args, **kwargs)
File "/Users/clewis/dev/libprs500/trunk/src/libprs500/communicate.py", line 592, in free_space
command_number=FreeSpaceQuery.NUMBER)[0]
File "/Users/clewis/dev/libprs500/trunk/src/libprs500/communicate.py", line 389, in _bulk_read
packet = bulk_read_packet(data_type=data_type, size=packet_size)
File "/Users/clewis/dev/libprs500/trunk/src/libprs500/communicate.py", line 379, in bulk_read_packet
data = data_type(self.handle.bulk_read(self.BULK_IN_EP, rsize))
File "/Users/clewis/dev/libprs500/trunk/src/libprs500/prstypes.py", line 717, in __init__
" packets must have a length of atleast 16 bytes")
libprs500.errors.PacketError: 'libprs500.prstypes.FreeSpaceAnswer packets must have a length of atleast 16 bytes

I wish I had access to a machine running OSX. Does the following python script cause an error? Also I added some extra error reporting to the gui. So update and try again.


from libprs500.communicate import PRS500Device

dev = PRS500Device()
print dev.get_device_information(end_session=False)
print dev.free_space(end_session=False)

I wish I had access to a machine running OSX. Does the following python script cause an error? Also I added some extra error reporting to the gui. So update and try again.

No errors on the test script (hooray!).

GUI says:

1 Command Type: GetUSBProtocolVersion
00000000: 0000 0000 0100 0000 0000 0000 0400 0000 ................
00000010: 0000 0000 ....
--
2 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
--
Traceback (most recent call last):
File "/Users/clewis/dev/Reader/libprs500/trunk/src/libprs500/gui/main.py", line 523, in establish_connection
info = self.dev.get_device_information(end_session=False)
File "/Users/clewis/dev/Reader/libprs500/trunk/src/libprs500/communicate.py", line 167, in run_session
dev.open()
File "/Users/clewis/dev/Reader/libprs500/trunk/src/libprs500/communicate.py", line 259, in open
version = self._bulk_read(24, data_type=USBProtocolVersion)[0].version
File "/Users/clewis/dev/Reader/libprs500/trunk/src/libprs500/communicate.py", line 392, in _bulk_read
packet = bulk_read_packet(data_type=data_type, size=packet_size)
File "/Users/clewis/dev/Reader/libprs500/trunk/src/libprs500/communicate.py", line 379, in bulk_read_packet
data = data_type(self.handle.bulk_read(self.BULK_IN_EP, rsize))
File "/Users/clewis/dev/libprs500/trunk/src/libprs500/prstypes.py", line 717, in __init__
PacketError: 'libprs500.prstypes.USBProtocolVersion packets must have a length of atleast 16 bytes
Unable to connect to device. Please try unplugging and reconnecting it

Pity. The PyQt tarball for OSX doesn't compile?

It does compile, once you get the other issues out of the way (see my previous posts). I've been travelling and haven't been able to get back to try out the GUI. I'm psyched, however, by the progress reported here in recent days!

Xenophon

I wish I had access to a machine running OSX. Does the following python script cause an error? Also I added some extra error reporting to the gui. So update and try again.


from libprs500.communicate import PRS500Device

dev = PRS500Device()
print dev.get_device_information(end_session=False)
print dev.free_space(end_session=False)



The test script works fine for me too. Running the GUI gets me the following (with packet logging on):
1 Command Type: GetUSBProtocolVersion
00000000: 0000 0000 0100 0000 0000 0000 0400 0000 ................
00000010: 0000 0000 ....
--
2 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
--
3 Answer d->h Type: USBProtocolVersion
00000000: 0500 0100 0000 0000 0000 0000 0800 0000 ................
00000010: 3031 3030 3030 3030 01000000
--
4 Command Type: AcknowledgeBulkRead
00000000: 0010 0000 0000 0000 0000 0000 1000 0000 ................
00000010: 0500 0100 0000 0000 0000 0000 0000 0000 ................
--
5 Response Type: Response
00000000: 0010 0000 0000 0000 0000 0000 0c00 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
--
6 Command Type: SetBulkSize
00000000: 0701 0000 0100 0000 0000 0000 0400 0000 ................
00000010: 0080 0200 ....
--
7 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 0701 0000 0000 0000 0000 0000 0000 0000 ................
--
8 Command Type: UnlockDevice
00000000: 0601 0000 0100 0000 0000 0000 0400 0000 ................
00000010: 2d31 0000 -1..
--
9 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 0601 0000 0000 0000 0000 0000 0000 0000 ................
--
10 Command Type: SetTime
00000000: 0401 0000 0100 0000 0000 0000 1c00 0000 ................
00000010: d4fe ffff d707 0000 0100 0000 1900 0000 ................
00000020: 0400 0000 2000 0000 2500 0000 .... ...%...
--
11 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 0401 0000 0000 0000 0000 0000 0000 0000 ................
--
12 Command Type: DeviceInfoQuery
00000000: 0101 0000 0100 0000 0000 0000 0000 0000 ................
--
13 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 1000 0000 ................
00000010: 0101 0000 0000 0000 0400 0000 5801 0000 ............X...
--
14 Answer d->h Type: DeviceInfo
00000000: 0000 1000 0000 0000 0000 0000 5801 0000 ............X...
00000010: 536f 6e79 2052 6561 6465 7200 0000 0000 Sony Reader.....
00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000030: 5052 532d 3530 302f 5500 0000 0000 0000 PRS-500/U.......
00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000050: 312e 302e 3030 2e32 3130 3831 0000 0000 1.0.00.21081....
00000060: 0000 0000 0100 0000 6170 706c 6963 6174 ........applicat
00000070: 696f 6e2f 782d 6262 6562 2d62 6f6f 6b00 ion/x-bbeb-book.
00000080: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000090: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000100: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000110: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000120: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000130: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000140: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000150: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000160: 0000 0000 0000 0000 ........
--
15 Command Type: AcknowledgeBulkRead
00000000: 0010 0000 0000 0000 0000 0000 1000 0000 ................
00000010: 0000 1000 0000 0000 0000 0000 0000 0000 ................
--
16 Response Type: Response
00000000: 0010 0000 0000 0000 0000 0000 1000 0000 ................
00000010: 0101 0000 0000 0000 0400 0000 5801 0000 ............X...
--
17 Command Type: FreeSpaceQuery
00000000: 0301 0000 0100 0000 0000 0000 0400 0000 ................
00000010: 0000 0000 ....
--
18 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 0301 0000 0000 0000 0000 0000 0000 0000 ................
--
release_interface() takes exactly 2 arguments (1 given)
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-pac
kages/libprs500-0.3.5-py2.5.egg/libprs500/gui/main.py", line 538, in establish_c
onnection
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-pac
kages/libprs500-0.3.5-py2.5.egg/libprs500/gui/main.py", line 552, in update_avai
labe_space
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-pac
kages/libprs500-0.3.5-py2.5.egg/libprs500/communicate.py", line 168, in run_sess
ion
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-pac
kages/libprs500-0.3.5-py2.5.egg/libprs500/communicate.py", line 575, in free_spa
ce
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-pac
kages/libprs500-0.3.5-py2.5.egg/libprs500/communicate.py", line 373, in _bulk_re
ad
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-pac
kages/libprs500-0.3.5-py2.5.egg/libprs500/communicate.py", line 363, in bulk_rea
d_packet
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-pac
kages/libprs500-0.3.5-py2.5.egg/libprs500/libusb.py", line 257, in bulk_read
libprs500.libusb.Error: Could not read 24 bytes on the bulk bus. Read: 0 bytes.



Seems like it's close to being there. I look forward to having things up and running!

Xenophon

This OSX bug is fixed in 0.3.6, thanks to chrissam letting me play with his OSX machine. Unfortunately, I don't understand the fix, which doesn't give me a lot of confidence when it comes to the rest of GUI. So please backup media.xml and cache.xml and test things.

The GUI now appears to run without problems on my Mac OS X 10.4.8 box (a 1.44GHz G4). I can list the books on the Reader, add books to the library, and so on.

A few questions:

Do you support copying from either the device's memory or the device's card back to the Library?
If yes, can you grab the metadata from the Reader and associate it with the new library entry?
Where is the library actually stored? Does the location differ on various operating systems?


The GUI is slick, and seems to work pretty well. It's also dog-slow on my machine. I'd be happy to produce whatever info might help with that (although I suspect it's more likely to be either an immature QT port or my old hardware that is primarily to blame). And, from the "give-em-an-inch-and-they-want-a-mile" department... When can we edit metadata from .rtf files? And when will you read the metadata that's already there? I've just been through the process of editing the metadata for 500 or so rtf files, and don't look forward to doing it again...

Xenophon

P.S. Thank you VERY much for your efforts so far. It is very cool to see an open source tool like this coming along so quickly!

Thanks, I'm glad to hear that it is working.

The GUI is pretty smooth on my 1.6GHz P4 laptop in both win32 and linux, so I'm inclined to believe that it's an OSX problem. Are there any other OSX users who can comment on this?

The library is stored in a file called library.db, its location is OS dependent, though at some point I will make it configurable.

Copying things from the reader to the library and/or your hard disk is on my todo list. It will transfer metadata automatically when it does that.

RTF and PDF support are also on my todo list, see the tickets on the libprs500 website for details.

Now that the basic GUI seems to be working on all platforms, I'll start working on more features. Unfortunately, I'm somewhat swamped in my day job, so progress is going to be slow for the next couple of months.

The GUI now appears to run without problems on my Mac OS X 10.4.8 box (a 1.44GHz G4). I can list the books on the Reader, add books to the library, and so on.

A few questions:

Do you support copying from either the device's memory or the device's card back to the Library?
If yes, can you grab the metadata from the Reader and associate it with the new library entry?
Where is the library actually stored? Does the location differ on various operating systems?


The GUI is slick, and seems to work pretty well. It's also dog-slow on my machine. I'd be happy to produce whatever info might help with that (although I suspect it's more likely to be either an immature QT port or my old hardware that is primarily to blame). And, from the "give-em-an-inch-and-they-want-a-mile" department... When can we edit metadata from .rtf files? And when will you read the metadata that's already there? I've just been through the process of editing the metadata for 500 or so rtf files, and don't look forward to doing it again...

Xenophon

P.S. Thank you VERY much for your efforts so far. It is very cool to see an open source tool like this coming along so quickly!

It works now on my MacBook (though strangely still not on my desktop; I'll clean things out and try again). Thanks Kovid!

Hi, and thank you Kovid for all of this work! This question may have already been addressed in another form (I think maybe chrissam had a similar problem above) but I've installed everything and yet prs500 won't recognize my reader, at least not in any meaningful way--I assume this has something to do with the USB connect via OSX, since I get different errors when entering the command several times within a short span (though the errors are different from chrissam's)... I'm working on a new MacBook Core2duo if that helps anything.

Here's the most common error message:

Books in main memory:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/Current/bin/prs500", line 8, in <module>
load_entry_point('libprs500==0.3.6', 'console_scripts', 'prs500')()
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.6-py2.5.egg/libprs500/cli/main.py", line 210, in main
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.6-py2.5.egg/libprs500/communicate.py", line 167, in run_session
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.6-py2.5.egg/libprs500/communicate.py", line 256, in open
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.6-py2.5.egg/libprs500/communicate.py", line 317, in send_validated_command
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.6-py2.5.egg/libprs500/communicate.py", line 301, in _send_command
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.6-py2.5.egg/libprs500/prstypes.py", line 643, in __init__
libprs500.errors.PacketError: Response packets must have their number set to 0x1000

though this one pops up too:

prs500 books
prs500 booksBooks in main memory:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/Current/bin/prs500", line 8, in <module>
load_entry_point('libprs500==0.3.6', 'console_scripts', 'prs500')()
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.6-py2.5.egg/libprs500/cli/main.py", line 210, in main
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.6-py2.5.egg/libprs500/communicate.py", line 167, in run_session
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.6-py2.5.egg/libprs500/communicate.py", line 268, in open
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.6-py2.5.egg/libprs500/communicate.py", line 317, in send_validated_command
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.6-py2.5.egg/libprs500/communicate.py", line 301, in _send_command
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libprs500-0.3.6-py2.5.egg/libprs500/libusb.py", line 229, in control_msg
libprs500.libusb.Error: Could not read 32 bytes on the control bus. Read: -60 bytes.

Any suggestions? I've tried "hammering" the command over and over to get the USB to say something more interesting, but to no avail. I'm new to this so it might be totally easy and stupid, but: Any suggestions?

Thanks!
Mikey P.

Hmm when you get this type of error, try reconnecting the reader. (Unplug and reconnect it). Also do you always get errors or are they intermittent?

Yeah, I tried reconnecting it (in any number of comically hopeful/absurd ways), but it happens all the time. Do those errors mean that my mac's simply not even recognizing the reader at all, or is it something else?

No its finding the reader, the reader is just not responding like it should. Can you try the commands with the --log-packets option.

Thanks Kovid, I tried it with the --log-packets option, and it logged some packets, one of which mentioned unlocking the reader... which made me think to turnoff the screen-lock on the Reader (under Settings). Then it took a couple times to find the Reader, and then it worked.

OK, now to embark on the gui...

I've taken a look at the RTF spec and added support for reading RTF metadata in SVN. (There's an rtf-meta command). However, it's going to have problems with non-english characters.

I've also decided not to implement write support for the metadata as RTF is a bloody mess and I don't feel like writing a full-fledged RTF parser. You don't really need metadata support as if you transfer files to the reader via the GUI, the GUI automatically sets the title and author.

If however someone feels they really need RTF metadata write support, I encourage them to look at rtf.py and extend it.

I would appreciate some testing from those of you that use RTF. It worked fine for the few RTF files I have.

Thanks Kovid, I tried it with the --log-packets option, and it logged some packets, one of which mentioned unlocking the reader... which made me think to turnoff the screen-lock on the Reader (under Settings). Then it took a couple times to find the Reader, and then it worked.

OK, now to embark on the gui...

I've added support for unlocking the device in version 0.3.8

I've taken a look at the RTF spec and added support for reading RTF metadata in SVN. (There's an rtf-meta command). However, it's going to have problems with non-english characters.
You rock! I'll try tonight one more time to compile the gui on my Ubuntu installation.

I've taken a look at the RTF spec and added support for reading RTF metadata in SVN. (There's an rtf-meta command). However, it's going to have problems with non-english characters.

I've also decided not to implement write support for the metadata as RTF is a bloody mess and I don't feel like writing a full-fledged RTF parser. You don't really need metadata support as if you transfer files to the reader via the GUI, the GUI automatically sets the title and author.

If however someone feels they really need RTF metadata write support, I encourage them to look at rtf.py and extend it.

I would appreciate some testing from those of you that use RTF. It worked fine for the few RTF files I have.

I don't think I need metadata write support for RTF, I just want to avoid re-entering all the metadata I just fixed up. I'll try out the latest version and let you know how it does.

After the upgrade... Macbook Pro Mac OS X 10.4.8...

scotty$ prs500 --log-packets ls /
1 Command Type: GetUSBProtocolVersion
00000000: 0000 0000 0100 0000 0000 0000 0400 0000 ................
00000010: 0000 0000 ....
--
2 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
--
3 Answer d->h Type: USBProtocolVersion
00000000: 0500 0100 0000 0000 0000 0000 0800 0000 ................
00000010: 3031 3030 3030 3030 01000000
--
4 Command Type: AcknowledgeBulkRead
00000000: 0010 0000 0000 0000 0000 0000 1000 0000 ................
00000010: 0500 0100 0000 0000 0000 0000 0000 0000 ................
--
5 Response Type: Response
00000000: 0010 0000 0000 0000 0000 0000 0c00 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
--
6 Command Type: SetBulkSize
00000000: 0701 0000 0100 0000 0000 0000 0400 0000 ................
00000010: 0080 0200 ....
--
7 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 0701 0000 0000 0000 0000 0000 0000 0000 ................
--
8 Command Type: UnlockDevice
00000000: 0601 0000 0100 0000 0000 0000 0400 0000 ................
00000010: 2d31 0000 -1..
--
9 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 0601 0000 0000 0000 0000 0000 0000 0000 ................
--
10 Command Type: SetTime
00000000: 0401 0000 0100 0000 0000 0000 1c00 0000 ................
00000010: 20fe ffff d707 0000 0200 0000 0900 0000 ...............
00000020: 1200 0000 3300 0000 0f00 0000 ....3.......
--
11 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 0401 0000 0000 0000 0000 0000 0000 0000 ................
--
12 Command Type: PathQuery
00000000: 1800 0000 0100 0000 0000 0000 0500 0000 ................
00000010: 0100 0000 2f ..../
--
13 Response Type: ListResponse
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 1800 0000 d6ff ffff 0000 0000 0000 0000 ................
--
14 Answer d->h Type: FileProperties
00000000: 0300 0100 0000 0000 0000 0000 1800 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000020: 0000 0000 0000 0000 ........
--
15 Command Type: AcknowledgeBulkRead
00000000: 0010 0000 0000 0000 0000 0000 1000 0000 ................
00000010: 0300 0100 0000 0000 0000 0000 0000 0000 ................
--
16 Response Type: Response
00000000: 0010 0000 0000 0000 0000 0000 0c00 0000 ................
00000010: 1800 0000 d6ff ffff 0000 0000 0900 0000 ................
--
17 Command Type: EndSession
00000000: 0100 0000 0100 0000 0000 0000 0400 0000 ................
00000010: 0000 0000 ....
--
18 Response Type: Response
00000000: 0010 0000 0100 0000 0000 0000 0c00 0000 ................
00000010: 0100 0000 0000 0000 0000 0000 0000 0000 ................
--
has an unknown error. Code: 0xffffffd6L

My prs500 FUSE filesystem is still working...

Hmm make that my regression tests were working. They evidently don't like a path query on / anymore. I like that nice clean file property they're returning now though. :D

Good news! From discussions with them both at the Bloggers Day and the Book Festival in October, I'm not really surprised that they didn't lock you out, but it's good to have it confirmed, regardless. :yes:

Looks like they deny permission for ls, get requests for any path that doesn't begin with /Data. Doesn't break ebook transfer. And I suspect that you just have to unlock the rest via a new USB command. Unfortunately, I don't have the time right now.

You are correct.

I was in the process of tweaking my code to have 3 directories (based on feed back from my wife):

Root
Internal
Card

I guess I'll just have two for now. :)