kindle touch hacks for handicapped people

[pirl8]

A friend of mine suffers from "low vision" due to "Retinitis pigmentosa".
He can barely read a monitor and only if he so close that his nose actually touches the glass.

He still uses a CRT monitor because the side angle of LCD is always too low watching so close. He sees a small circle with text with borders rapidly fading to black.

I let him try my Kindle touch. The good part is that can read it well and that he enjoys font resizing. He loves it and he would like an e-ink monitor for PC.

However there are two main problems:
1) since he needs to stay so close the display is too dark (he shades the light with his head). An external light is annoying because it refletcs on his glasses.
2) he touches the display with his nose, thus randomly changing pages, and so on

The ideal for him would be a non-touch kindle with the paperwhite illuminated display.

Maybe a Kindle touch for handicapped people should let them choose from different layouts for turning pages. In the case of my friend the best would be to turn pages only sliding on the bottom of the screen.

[knc1]

Intercepting the capacative touch screen events should be possible.
This would give the effect of a "non-touch" Kpw.

Intercepting the capacative touch screen events to detect: Nose down, Nose up, Nose left, Nose right, etc might be difficult, might be **very** difficult depending on the code path from the event inputs.

Here: "Nose ***" would be the same as "long touch" or "long tap".

[ixtab]

@toka: Whoa... that's really an impressive wishlist, so let's get things straight:
What you are asking for is almost a completely new firmware, not a modification of the existing one. I'm afraid this can't be done, at least not on a hobby basis.

The one crucial thing is that it's almost impossible, without MAJOR changes, to change the functionality of the home button. Home button = start home screen booklet. Nothing else.

The solution that I proposed above would require a patch to the reader (a relatively small one, but it's still quite a bit of work to find out how exactly it has to be implemented), and a substitute for the home screen booklet (which would have to be written from scratch, but that is -mostly- standard coding).

What I can offer is to take a look into the abovementioned two topics first. No guarantees or deadlines though. Once that works ok, we could still get back to see whether it's feasible to include some of your suggestions concerning the home button (or, for instance, "key presses with indicator"). Interested?

[eureka]

Originally Posted by ixtab

The one crucial thing is that it's almost impossible, without MAJOR changes, to change the functionality of the home button. Home button = start home screen booklet. Nothing else.

Well, it could be changed without major changes, but it will possibly require knowledge of Lua. Though, Lua is easy language (yes, really).

Home button action is defined in /etc/xdg/awesome/rc.lua in function handleHomeKeyUp (there is also small partner function handleHomeKeyDown). At the end of function, there is following chunk of code:

-- launch the default app                                   
log("==============Sending Home Button")                    
logTimeStamp("home button press")                           
client.validate_stacking_order()                            
lipcH:set_int_property("com.lab126.appmgrd", "startdefault", 0)

So, that's how it works.

lipcH is a top-scope variable (defined in rc.lua) with handle for working with LIPC. In fact, one could not only set any external property through it, but also register new property for provider com.lab126.winmgr and change value of this new property appropriately. And even subscribe to LIPC events. Or send such events. For examples and insights:

Spoiler:

grep -r lipcH /etc/xdg/awesome

strings /usr/lib/lua/liblipclua.so

So, with the help of LIPC one can transimt status of Home button from rc.lua to outside world. Or just read it's state from some other source (physical Home key is mapped as keyboard Home key).

Sorry if it sounded like a crude refutation, I just wanted to help a little.

[ixtab]

Originally Posted by eureka

Well, it could be changed without major changes, but it will possibly require knowledge of Lua. Though, Lua is easy language (yes, really).

Home button action is defined in /etc/xdg/awesome/rc.lua in function handleHomeKeyUp (there is also small partner function handleHomeKeyDown). At the end of function, there is following chunk of code:

-- launch the default app                                   
log("==============Sending Home Button")                    
logTimeStamp("home button press")                           
client.validate_stacking_order()                            
lipcH:set_int_property("com.lab126.appmgrd", "startdefault", 0)

So, that's how it works.

lipcH is a top-scope variable (defined in rc.lua) with handle for working with LIPC. In fact, one could not only set any external property through it, but also register new property for provider com.lab126.winmgr and change value of this new property appropriately. And even subscribe to LIPC events. Or send such events. For examples and insights:

Spoiler:

grep -r lipcH /etc/xdg/awesome

strings /usr/lib/lua/liblipclua.so

So, with the help of LIPC one can transimt status of Home button from rc.lua to outside world. Or just read it's state from some other source (physical Home key is mapped as keyboard Home key).

Sorry if it sounded like a crude refutation, I just wanted to help a little.

Yep, I knew about most of this (not about the LIPC part though, thanks for pointing that out).

I had already thought about posting a LIPC event on button press... then of course any software interested in it (that would include at least the home & reader booklets) would have to be changed to listen for these events, and to act accordingly.

But this is what I consider to be a "major" change

[eureka]

Originally Posted by ixtab

Yep, I knew about most of this (not about the LIPC part though, thanks for pointing that out).

I had already thought about posting a LIPC event on button press... then of course any software interested in it (that would include at least the home & reader booklets) would have to be changed to listen for these events, and to act accordingly.

But this is what I consider to be a "major" change

OK, then a small question (made out of curiosity): did you ever tried to comment that lipcH:set_int_property line, restart X server, then check if physical Home key is recognized as keyboard Home key in kindlet/booklet (through using of KeyListener and VK_HOME or something similar)?

(I didn't and my KT is a little bricked at the time)

UPD Oh, line to comment should be

awful.key({ "Any",      }, "Home", handleHomeKeyDown, handleHomeKeyUp),

(it's a bit screens down from handleHomeKeyUp function)

[ixtab]

Originally Posted by eureka

OK, then a small question (made out of curiosity): did you ever tried to comment that lipcH:set_int_property line, restart X server, then check if physical Home key is recognized as keyboard Home key in kindlet/booklet (through using of KeyListener and VK_HOME or something similar)?

(I didn't and my KT is a little bricked at the time)

Nope, not yet - as said, I wasn't aware about the possibility to directly use LIPC from the LUA script. I only knew that it has the command to start the home booklet hardcoded. I can't try it either atm, don't have the device around...

[eureka]

Originally Posted by toka

Is it possible to use the usb port as host ?
This would allow to connect a custom interaface by cable or bluetooth, which could be optimized for the persons special needs.

It's possible but not in stock software configuration. Some OS parts should be re-worked, some should be ported. And no-one at the moment take any initiative in this field. I.e. no easy way.

[eureka]

Originally Posted by ixtab

Nope, not yet - as said, I wasn't aware about the possibility to directly use LIPC from the LUA script. I only knew that it has the command to start the home booklet hardcoded. I can't try it either atm, don't have the device around...

Uhhmmm... But point of my previous post wasn't in using LIPC from Lua. I'm saying about using of KeyListener with VK_HOME for catching Home button press straight from Java code.

Anyway, it doesn't matter a lot as you are also can't try right now

[eureka]

Originally Posted by eureka

OK, then a small question (made out of curiosity): did you ever tried to comment that lipcH:set_int_property line, restart X server, then check if physical Home key is recognized as keyboard Home key in kindlet/booklet (through using of KeyListener and VK_HOME or something similar)?

(I didn't and my KT is a little bricked at the time)

UPD Oh, line to comment should be

awful.key({ "Any",      }, "Home", handleHomeKeyDown, handleHomeKeyUp),

(it's a bit screens down from handleHomeKeyUp function)

So, I've checked it and it's indeed enough to comment this line in /etc/xdg/awesome/rc.lua (and then restart x) to disable default Home button action.

Then I've also made a little kindlet with JTextArea and Home button didn't return cursor to starting position in editing mode. So (with my limited knowledge of Java and Kindlet system) I can say that Java susbsytem can't react to Home button as to keyboard button. However, I can be wrong.

Still, kernel sources and evtest /dev/input/by-path/platform-whitney-button-event confirms that KT's Home button has keycode of ordinary keyboard Home button.

[eureka]

Originally Posted by eureka

So, I've checked it and it's indeed enough to comment this line in /etc/xdg/awesome/rc.lua (and then restart x) to disable default Home button action.

Then I've also made a little kindlet with JTextArea and Home button didn't return cursor to starting position in editing mode. So (with my limited knowledge of Java and Kindlet system) I can say that Java susbsytem can't react to Home button as to keyboard button. However, I can be wrong.

Still, kernel sources and evtest /dev/input/by-path/platform-whitney-button-event confirms that KT's Home button has keycode of ordinary keyboard Home button.

Just a note: there are files /usr/java/lib/keys and /usr/java/lib/keys_alt which are looked like whitelist of keys that can be passed from X server to Java app and also a mapping of X keycode to Java keycode. And there is no entry for Home button in these files.

[twobob]

hmm. interesting info. poking around in the X config is fun. Thanks

[baf]

I want to add my brute force contribution to this topic. I wrote a very simple program, whose main aim is to turn Kindle Touch into a Non-Touch .

It just grabs all X events – keyboard (read One Key) and touchscreen input, so that the events don't reach other apps. It is just a proof of concept which may be further developed or used for some other project.

I adjusted the program to what was asked for in one of the previous posts in this thread. It disables touchscreen input and turns home button events into touchscreen events, so that home button may be used to turn pages in reader application.
When started program waits for pressing home button. When you press the home button, the touchscreen becomes inactive. There is only small 50×50 pixels active area in the upper right corner of the screen. Touching this area turns the program off.
Home button presses are translated into touchscreen clicks. Short button press simulates click on the right side of the screen – forward page turn. Long button press imitates click on the left side of the screen – backward page turn. Duration of the long press may be adjusted on command line.

I also prepared a GUI Launcher extension for those who don't want to mess around with command line. To use for reading a book start the extension from extensions folder. Then click on the book you want to read and when it opens just press home button. From this moment home button will work as described above and touchscreen will not work. To turn it off you will need to click in the upper right corner of the screen.

You will find everything on my webpage.

[root@kindle bin]# ./untouchable -h
Usage: untouchable [OPTION]
	-t <milliseconds> - minimal time home button should be hold to be treated as long press (default 1500)
	-d               - debug
	-h               - show this message

untouchable.c:

Spoiler:

/*  
    untouchable - filter all input and make Kindle Touch "untouchable" :)
    Copyright (c) 2012 by baf (www.fabiszewski.net), with MIT license:
    http://www.opensource.org/licenses/mit-license.php

    gcc untouchable.c -O2 -Wall -lX11 -o untouchable
*/

#include <stdio.h> // printf
#include <stdbool.h> // bool
#include <string.h> // memset
#include <stdlib.h> // strtol
#include <unistd.h> // usleep
#include <X11/keysym.h>
#include <X11/XKBlib.h>

void pageForward(Display *display);
void pageBackward(Display *display);
void SendMouseEvent(Display *display, int button, int x, int y, bool pressed);
void usage();

bool debug = false;

int main(int argc, char * argv[]) {
	Window root;
	Display * display;
	XEvent xev;
	Time clickTime = CurrentTime;
	int longPressDuration = 1500;
	int timeDiff;
	int screenWidth;
	
	// parse options
	while ((argc > 1) && (argv[1][0] == '-')){
		switch (argv[1][1]){
			case 't':
				++argv;
				if (--argc <= 1){
					printf("Option -t requires numeric argument\n");
					usage();
					return 1;
				}
				longPressDuration = strtol(argv[1],0,0); 
				break;
                        case 'd':
				debug = true;
                                break;
                        case 'h':
				usage();
                                return 0;
		}
		++argv;
		--argc;
	}

	if(debug) printf("Long press duration: %i\n", longPressDuration);

	display = XOpenDisplay(NULL);
	if(display == NULL) {
		if(debug) printf("Could not open display\n");
		return 1;
	}

	root = DefaultRootWindow(display);
	screenWidth = WidthOfScreen(DefaultScreenOfDisplay(display));
	if(debug) printf("Screen width: %i\n", screenWidth);

	XAllowEvents(display, AsyncBoth, CurrentTime);
	// grab key events
	if(debug) printf("Grab key events\n");
	XGrabKeyboard(display, root, true, GrabModeAsync, GrabModeAsync, CurrentTime); 

	bool go = true;
	bool grabOn = false; // we start grabbing mouse events only after first press of home button
	while(go){
		XNextEvent(display, &xev);
	        int keycode = (int) XLookupKeysym(&xev.xkey, 0);

		switch(xev.type){
			case ButtonPress:
				if(debug) printf("Button press - %i\n", xev.xbutton.button);
				if(debug) printf("Coordinates - %i, %i\n", xev.xbutton.x, xev.xbutton.y);
			break;

			case ButtonRelease:
				if(debug) printf("Button release - %i\n", xev.xbutton.button);
				if(xev.xbutton.x>screenWidth-50 && xev.xbutton.y<50)
					go = false;
			break;
			case KeyPress:
				clickTime = xev.xkey.time;
				if(debug) printf("Key press - %i\n", keycode);
			break;
			case KeyRelease:
				timeDiff = (long) xev.xkey.time - (long) clickTime;
				if(debug) printf("Key release - %i\n", keycode);
				if(debug) printf("Key press duration - %i\n", timeDiff);
				// FIXME: test for keycode is rather useless :)
			        if(keycode == XK_Home && grabOn){
					if(timeDiff < longPressDuration)
						pageForward(display);
					else
						pageBackward(display);
				} else {
					// first press of home button
					if(debug) printf("Grab mouse events\n");
					grabOn = true;
					XGrabButton(display, AnyButton, AnyModifier, root, true, ButtonPressMask | ButtonReleaseMask, GrabModeAsync, GrabModeAsync, None, None);
				}
			break;
		}
	}

	XUngrabKeyboard(display, CurrentTime);
	if(grabOn) XUngrabButton(display, AnyButton, AnyModifier, root);

	XCloseDisplay(display);

	return 0;
}

void pageForward(Display *display){
	SendMouseEvent(display, Button1, 500, 300, true);
	usleep(10000);
	SendMouseEvent(display, Button1, 500, 300, false);
}

void pageBackward(Display *display){
        SendMouseEvent(display, Button1, 50, 300, true);
        usleep(10000);
        SendMouseEvent(display, Button1, 50, 300, false);
}

void SendMouseEvent(Display *display, int button, int x, int y, bool pressed){
	XEvent event;
	memset(&event, 0, sizeof(event));
	event.xbutton.type = pressed ? ButtonPress : ButtonRelease;
	event.xbutton.button = button;
	event.xbutton.same_screen = true;
	XQueryPointer(display, RootWindow(display, DefaultScreen(display)), 
		&event.xbutton.root, &event.xbutton.window, &event.xbutton.x_root, 
		&event.xbutton.y_root, &event.xbutton.x, &event.xbutton.y, &event.xbutton.state);
	
	event.xbutton.subwindow = event.xbutton.window;
	
	while(event.xbutton.subwindow){
		event.xbutton.window = event.xbutton.subwindow;
		
		XQueryPointer(display, event.xbutton.window, 
			&event.xbutton.root, &event.xbutton.subwindow, &event.xbutton.x_root, 
			&event.xbutton.y_root, &event.xbutton.x, &event.xbutton.y, &event.xbutton.state);
	}
	event.xbutton.x = x;
	event.xbutton.y = y;
	if(debug) printf("Sending %i %i event: %ix%i\n", event.xbutton.type, event.xbutton.button, x, y); 

	if (!XSendEvent(display, PointerWindow, true, 0xfff, &event))
		printf("Failed to send mouse event\n");
	XFlush(display);
}

void usage(){
	printf("Usage: untouchable [OPTION]\n");
	printf("	-t <milliseconds> - minimal time home button should be hold to be treated as long press (default 1500)\n");
	printf("	-d               - debug\n");
	printf("	-h               - show this message\n");
}

[eureka]

Originally Posted by baf

It just grabs all X events – keyboard (read One Key) and touchscreen input, so that the events don't reach other apps.

Ooooh, what a simple sexy idea! Genius move. Thanks for sharing.

[Darqref]

Originally Posted by baf

It just grabs all X events – keyboard (read One Key) and touchscreen input, so that the events don't reach other apps. It is just a proof of concept which may be further developed or used for some other project.

Do any of the Kindle Touch devices still have TTS? I'd love to be able to use this type of interception to turn on/off TTS. Assuming book is already selected, or using the same type of key interception described above to move along a short list of books, then start reading and then turn on TTS.

(Mom is legally blind, might be able to see a title at one per page. So far, the idea of needing to hit two separate buttons (on the KK) at the same time to turn on TTS has been discouraging.)