Device detection process

[denverh]

Hello,

I'm going to try adding the functions that are missing from the FreeBSD port. Those are all involved in detecting and mounting the reader when it's plugged in. It would help a bit if I understood the process for the Linux environment. It looks to me like /sys/devices is scanned for anything that matches the list of vendor and product IDs, but I'm not sure how this gets started, or where it goes from there. That's about as far as I've gotten so far. I don't have much experience with python, so this is going to be an interesting project.

If someone could describe, even briefly, the existing device detection and mounting process for Linux, I would greatly appreciate it.

Thanks,

Denver

June 14, 2010
Ok, I've fixed up the mount helper so it functions properly, and added some code to /devices/usbms/device.py to mount and eject the reader. Everything _seems_ to be working the way it should. The code in device.py is pretty basic right now: I hard-coded the device node and mount points. But that's just to see if everything else is working, and if I was missing anything. Here's the part I added for mounting the device (I shamelessly borrowed from the linux section):

  def open_freebsd(self):
                node="/dev/da1"
                label="READER"
                cmd = '/usr/local/bin/calibre-mount-helper'
                cmd = [cmd, 'mount']
                try:
                        p = subprocess.Popen(cmd + [node, '/media/'+label])
                except OSError:
                        raise DeviceError(
                        _('Could not find mount helper: %s.')%cmd[0])
                while p.poll() is None:
                        time.sleep(0.1)
                self._main_prefix = '/media/READER'
                self._card_a_prefix = None
                self._card_b_prefix = None

Is there anything else that's going to need to be done in there, besides the obvious? If not, then it looks like the next thing is to figure out a way to discover the device nodes.

Thanks,

Denver

[kovidgoyal]

The class LinuxScanner in scanner.py is what scans /sys and returns a tuple of

(vendor id, product id, bcd, manufacturer, product name, serial number)

This tuple is used by the is_usb_connected methods defined in devices/interface.py to check if a device class matches one of the tuples returned by the scanner.

is_usb_connected also uses can_handle() which is typicall overridden in actual device drivers.

Finally, when an device is found its open() method is called which is responsible for mounting the device and setting the paths to the main memory/storage cvards of the device. See open_linux() in usbms/device.py

[denverh]

Thanks Kovid, that helps. The existing port detects and identifies readers without any problem. However, they don't get mounted, and of course no data is transferred. From your description of the process, the failure is in the open(). I've created a new open_freebsd() that will get used instead of falling through all the checks in open(). I've figured out a lot of what open_linux() is doing, but parts of it are still puzzles.

One piece of the puzzle that I'm still missing is where the vendor ID, product ID, etc. is kept. How do I get to that information from my open_freebsd()?

I also want to verify that all open_freebsd() needs to do is verify the device node, or nodes, against product & vendor IDs (or perhaps just S/N?), mount the device node, or nodes, and publish that by setting main_prefix, card_a_prefix, and card_b_prefix appropriately. Is there anything else that open_freebsd() needs to do?

And one last thing: you mentioned that there's more information available about the detected reader than vendor/product IDs and S/N. Is any of the rest of that information important to what I'm doing with open_freebsd()?


Thanks again,

Denver

[kovidgoyal]

self.detected_device see find_device_nodes() for an example.

Yes all you have to do is populate the *_prefix variables.

No, that should be all you need.

[denverh]

Thanks Kovid, that's what I needed. I have this working now. It's the first Python project I've undertaken, so it's probably not too elegant, but it all seems to be working the way it should.

Thanks again,

Denver

[kovidgoyal]

Cool, send me the patch.

[denverh]

I will be happy to, but there are some things I just noticed that may need some attention. First, it looks like the mount points returned in main_prefix, etc., need to end with "/". Is that correct? Right now mine don't, and everything was working, but I ran into a little trouble with one plugin.

Secondly, I see a number of errors reported if I run "calibre-debug -g" when my reader is plugged in. I can't see that anything is going wrong because of them, but I also can't see why they are happening. They all appear to be repeats of the following:

Traceback (most recent call last):
  File "/usr/local/lib/calibre/calibre/gui2/update.py", line 44, in run
    version = get_newest_version()
  File "/usr/local/lib/calibre/calibre/gui2/update.py", line 28, in get_newest_version
    version = br.open(req).read().strip()
  File "build/bdist.freebsd-8.2-STABLE-amd64/egg/mechanize/_mechanize.py", line 203, in open
    return self._mech_open(url, data, timeout=timeout)
  File "build/bdist.freebsd-8.2-STABLE-amd64/egg/mechanize/_mechanize.py", line 230, in _mech_open
    response = UserAgentBase.open(self, request, data)
  File "build/bdist.freebsd-8.2-STABLE-amd64/egg/mechanize/_opener.py", line 193, in open
    response = urlopen(self, req, data)
  File "build/bdist.freebsd-8.2-STABLE-amd64/egg/mechanize/_urllib2_fork.py", line 344, in _open
    '_open', req)
  File "build/bdist.freebsd-8.2-STABLE-amd64/egg/mechanize/_urllib2_fork.py", line 332, in _call_chain
    result = func(*args)
  File "build/bdist.freebsd-8.2-STABLE-amd64/egg/mechanize/_urllib2_fork.py", line 1142, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "build/bdist.freebsd-8.2-STABLE-amd64/egg/mechanize/_urllib2_fork.py", line 1118, in do_open
    raise URLError(err)
URLError: <urlopen error [Errno 4] Interrupted system call>

Can you suggest anything I can do to clean those up? They seem to be related to the subprocess.Popen() calls I'm making. If I make a version with everything hard-coded, and no subprocess.Popen(), then there are no errors.

Also, how do you want me to send the patches? As attachments here? There are also a few others besides mine that are necessary. I don't know if you have those or not.

I think mine are a good start, but probably need additional work.

Thanks,

Denver

[kovidgoyal]

Yes, you should end the *_prefix variables with a forward slash.

Those errors indicate that an attempt to either read or write or allocate a network socket is being interrupted. The most common cause of this is if the process has run out of available file handles. Make sure your subprocess calls close the stdout/stdin/stderr file handles immediately after using them.

EDIT: And you can attach the patches here or better, open a bug report and attach them to it.

[denverh]

Ok, thanks Kovid. It's easy enough to add the trailing fwd slashes.

I followed the examples in the python documentation for using subprocess, particularly when piping from one to another. The only places I saw a close for standard I/O was for the first stdout when piping to a second stdin. I duplicated that, but perhaps I missed something somewhere. I'll see if I can figure something out.

Thanks again,

Denver

[kovidgoyal]

You may be interested in:

https://launchpad.net/bugs/802708

[denverh]

Yes, those would be mine. Rusty is the port maintainer for Calibre on FreeBSD. I volunteered to help with getting the FreeBSD port to work with reader devices again. I'm happy to see that Rusty is passing the changes on.

Thanks,

Denver