Virtual frambuffer switching for Kobo

[fastrobot]

Hi,

There are several programs which run on the kobo reader and take it over, such as debian linux, or andriod, etc. And I was wondering if anyone had a program that is able to save the state of the framebuffer, eg; save the orientation, and data on the frame-buffer, and restore them -- so that switching between kobo reader mode, and these alternate systems could be done on the fly, rather than having to reboot and pull or insert an sd card?

I've written a virtual terminal that runs on my Kobo GLO, that I want to be able to run alternately with nickel; It's a vt52 emulator with unicode capability, international morse code keyboard interface that allows all control, alt, shift, keys and all characters on a vt52 keyboard to be produced without ever having to draw a vitual keyboard onscreen; including an extra xterm SGR mode mouse ( xterm mode 1006 ) which means full linux compatibilty for command line programs that run in an x-terminal under curses; eg: vim, emacs, etc. including cut and paste.

I set it up to proxy the touch-screen driver under /dev/input/event1 -- so that I can steal screen touches before nickel gets them and still pass them on to nickel if I don't want them; which allows me to catch an escape sequence that nickel happily ignores; so that I can use this gesture to launch a script to change modes between virtual terminal and nickel, and back again. I just need a program that can save and load the framebuffer so that I can complete the switch -- and I'd rather not re-invent it, if a program already exists.

Also, I have another issue,
I'd like my terminal program to be able to work with an extenal keyboard as well; and because of that, I bought an android external USB keyboard, and an adapter to plug in into kobo's USB OTG slot;
It does plug in just fine, but the KOBO kernel does not recognize the keyboard as it causes no dmesges at all -- Do I need to load a kernel driver, or something, and if so -- which one ?

[fastrobot]

It wasn't very difficult to write a framebuffer save and reload progarm, although I'm still having keyboard troubles; but if there already exists a program on the kobo -- let me know. Here's my version of a framebuffer save and restore if anyone else needs it for switching the framebuffer on the fly.

// fbswitch.c
//
// Strem new framebuffer in on stdin,
// Stream old framebuffer out on stdout,
// Include framebuffer configuration data when streaming framebuffer,
// so it may be restored identically.
// FIXME, maybe: IO restarting due to signal interruptions is not handled.h
//
// Examples of usage:
// echo -n | fbswitch | gzip - -c > /tmp/fb1.gz # only saves fb, no load.
// eg: zcat /tmp/fb0.gz | fbswitch | gzip - -c > /tmp/fb1.gz # save and load.

#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
#include <sys/ioctl.h>
#include <linux/fb.h>
#include "mxcfb.h"

int main(void) {
	int fbfd=open("/dev/fb0",O_RDWR);
	struct fb_var_screeninfo vinfo;
	struct mxcfb_update_data region;
	char* buffer[1024];
	int n;

	if (fbfd<0) return errno;
	errno=0;

	// First, backup the old framebuffer	
	ioctl(fbfd, FBIOGET_VSCREENINFO, &vinfo );
	if (errno) return errno;
	write( STDOUT_FILENO, (void*)&vinfo, sizeof(vinfo) ); // Save fb configure 
	while( (n=read(fbfd, buffer, 1024))>0 ) write(1, buffer, n);
	if (n<0) return errno;
	close(1);

	close(fbfd);
	fbfd = open("/dev/fb0",O_RDWR );

	// Next, load the framebuffer from stdin (if available).
	n=read( STDIN_FILENO, (void*)&vinfo, sizeof(vinfo));
	if (n && (n!=sizeof(vinfo))) return 1; // Corrupted configuration.. error. 
	if (n!=sizeof(vinfo)) return 0; // No data at all, so nothing to load.

	vinfo.rotate ^= 0x2; // Readback of the orientation is buggy, fix it.

	// Set the framebuffer mode, for the new picture.
	ioctl(fbfd, FBIOPUT_VSCREENINFO, &vinfo );
	if (errno) return errno;

	// Load the picture into the framebuffer.	
	while( (n=read(STDIN_FILENO,buffer,1024))>0 ) write( fbfd, buffer, n );
	if (n<0) return errno;

	// Refresh the screen with the new framebuffer.
	region.update_marker=0;
	region.update_region.top=0;
	region.update_region.left=0;
	region.update_region.width=vinfo.xres;
	region.update_region.height=vinfo.yres;	
	region.waveform_mode = WAVEFORM_MODE_AUTO;
	region.update_mode = UPDATE_MODE_FULL;
	region.temp = TEMP_USE_AMBIENT;
	region.flags=0;	
	ioctl( fbfd, MXCFB_SEND_UPDATE, &region );
	if (errno) return errno;
	ioctl( fbfd, MXCFB_WAIT_FOR_UPDATE_COMPLETE, &region.update_marker );
	return errno;
}

Edit: Note, the rotation orientation read back on the Kobo AuraHD is buggy and a fix is included to exclusive or it with a value to correct the defect. On Kobo Glos, though -- the bug does not exist; and so the above fix is a bad idea. You can either remove the line, and recompile -- or hex edit the binary and change the byte 0x02 to 0x00 right about file offset 0x490 or 0x491 (endianness issues), which effectively makes the EOR/XOR into a nop. ENDEdit.

I still can't figure out how to get the external keyboard to be recognized.
And, now that I have the screen redrawing -- I just noticed that nickel not only is ignoring the special mouse swipe -- it ignores all mouse gestures completely..

I edited the binary of libnickel, and pickel, so that the path to the touchscreen /dev/input/event1 was changed to /tmp/input/event1 -- and then made a named fifo at /tmp/input/event1 -- which echoes all the data from the mouse found on /dev/input/event1 -- but apparently nickel is doing something interactive with the touch device driver, or else is testing for the device node numbers or something; because when I run lsof on nickel, the file is not open to either /tmp/input or /dev/input... It closes both of them.

I don't see any error messages in dmesg, but I do see a reference to setting the mouse parameters -- and so I'm thinking nickel is doing an ioctl which is porbably failing when it does it on a fifo ?

[drivers/input/touchscreen/zforce_i2c.c-487] zforce_i2c_open()
[zForce_ir_touch_recv_data-175] command Activate (0) ...
[zForce_ir_touch_recv_data-184] command Resolution (0) ...
[zForce_ir_touch_recv_data-209] command Frequency (0) ...

Does anyone know how to find out for sure?
I tried to use strace on nickel in rcS -- and although it doesn't complain, the screen doesn't draw properly and I'm not getting a trace. Has anyone successfully straced nickel, and if so -- what command did you use, and where did you execute it ?

MD5Sums:
4b6fc11a58f88fca88f72b15186b45ed fbswitch.bz2
c8bbd94e1136865c939f76b43b79e2f9 fbswitch.c.bz2

[tshering]

Originally Posted by fastrobot

Also, I have another issue,
I'd like my terminal program to be able to work with an extenal keyboard as well; and because of that, I bought an android external USB keyboard, and an adapter to plug in into kobo's USB OTG slot;
It does plug in just fine, but the KOBO kernel does not recognize the keyboard as it causes no dmesges at all -- Do I need to load a kernel driver, or something, and if so -- which one ?

Maybe this thread has something of interest for you.

[tshering]

Originally Posted by fastrobot

Does anyone know how to find out for sure?
I tried to use strace on nickel in rcS -- and although it doesn't complain, the screen doesn't draw properly and I'm not getting a trace. Has anyone successfully straced nickel, and if so -- what command did you use,

The command I am using is

$ksmroot/tools/strace -t -ff -o $logfile /usr/local/Kobo/nickel -platform kobo -skipFontLoad

Originally Posted by fastrobot

and where did you execute it ?

I shifted the whole business of calling nickel (including exporting variables, calling on-animator.sh and so on) to another script and call it only after rcS is finished. Calling strace against nickel from telnet should also work.

[fastrobot]

Originally Posted by tshering

Maybe this thread has something of interest for you.

It does, indeed !

I did a diff on the native Kobo kernel config, and the one he is using.
There are more than just USB keyboard settings changed in his kernel, for example he's using the auto-update e-paper driver patch; which allows regular linux applications to run on the framebuffer -- but I think that would cause conflicts with nickel / QT doing manual updates; and since I need them to run together for right now; I don't want to replace the kernel yet.

There are three USB settings which he obviously changed, so I did a make menuconfig and added these options:

under: Device Drivers --> enabled[*] HID Devices (submenu enable), then
{M} Generic HID support
<M> USB Hum Interface Device (full HID) support
under: Device Drivers -> USB support --> [*] Support for DR host port on Freescale controller


But -- I think A keyboard runs in low-speed mode, so I'm thinking the last change is probably not necessary ? That's more of something needed to make external thumb drives work...

I'm hoping I can compile in the needed keyboard drivers as modules, and won't have to replace the kobo native kernel -- but only run insmod.

Hmmm... Now all I need is a power cord with USBOTG to USB adapter combo, as apparently these code changes are insufficient to get the OTG controller to supply power to the port -- but maybe that's just because he was using a USB hub ? ( EDIT: Just got one and tested, the keyboard didn't work, although the hid modules do load.).

There's probably a way to turn power on through the /sys interface, if I only knew the appropriate echo command to make it happen.... Well, guess I'll figure that out after I get everything else working since I know the keyboard can be operated with external power, I'll try that first.

....

I just discovered that udev doesn't like to have the 'info' mode set for debugging; eg: to test why a rule I made isn't being run.... and the rules do not appear to be in the default /etc/udev/rules.d directory; ugh! Udev behaves differently when debugging is attempted; and it also doesn't appear to be logging the info messages to the dmesg file. Anyone have an idea how udev is set up on an AuraHD? or where the log file is stored on a kobo?

I tested the android keyboard on a regular PC, and it came up fine; although my PC's dmesg log says hid-raw was involved. So I'll see if adding a kernel module for hid-raw helps. But on the KOBO the only message that appears is one saying "USB-plug" and "USB-unplug" when I insert and remove the keyboard. SO the hid modules are necessary, but not enough to make the keyboard work


MD5sums.
0f96250966d77aca5d3bbe44329d68d0 hid.ko.bz2
fefa81f74510f11251d01f954ca826d3 usbhid.ko.bz2

[NiLuJe]

IIRC, syslog doesn't log to a file on the Kobos, it only keeps track of the n'th latest events in memory, which you can access via logread.

No idea if udev even tries to communicate with the system logger properly on the Kobos, though.

[fastrobot]

Originally Posted by NiLuJe

IIRC, syslog doesn't log to a file on the Kobos, it only keeps track of the n'th latest events in memory, which you can access via logread.

No idea if udev even tries to communicate with the system logger properly on the Kobos, though.

Hmmm... it really doesn't look like it .
And the data in DMESG is actually different than that in logread.... I gave up, and worked the bug out a different way by running scripts on each rule, and having the scripts echo results to my own log file.

What was causing the problem is that I didn't realize that Udev no longer allows the changing of names of devices via the NAME variable... but I really need to change the actual name created in /dev/input, as nickel is hardwired to use a specific name. (I don't want to have to edit libnickel and pickel for future updates.)

And yes, I had to actually compile a dummy device driver (proxyevent) to act as a fifo so that nickel can do ioctls(), and read ABS ranges, etc. Apparently there is a user interface called 'uevent' that can make userspace driven dummy drivers -- but doing it that way was just as complicated (and probably longer code wise) than just duplicating the zforce kernel driver as a skeleton. So I just made a kernel driver because it was simpler.

I made UDEV replace the zforce driver with a dummy driver like this:

udev rule: 60-zforce-proxy.rules

ACTION=="add", KERNEL=="event*", SUBSYSTEMS=="input", ATTRS{name}=="zforce-ir-touch",  RUN+="/etc/udev/zforce-proxy.sh add %k"
ACTION=="remove", KERNEL=="event*", SUBSYSTEMS=="input", ATTRS{name}=="zforce-ir-touch", RUN+="/etc/udev/zforce-proxy.sh remove %k"

And script to actually swap out event1 with whatever the proxy is named....

#!/bin/sh
ORIGINAL="event1"
# Add and remove proxy from system

cd /dev/input
if [[ "$1" == "add" ]]; then
	if [[  -e $ORIGINAL && ! -h $ORIGINAL ]]; then
		mv $ORIGINAL zforce # Orignal renamed to zforce
		ln -s $2 $ORIGINAL  # Original now points at the proxy
		ln -s $2 zforce-proxy # Add a symlink for clarity
	fi
	exit 0
fi

if [[ "$1" == "remove" ]]; then
	if [[ -e zforce && ! -h zforce ]]; then
		mv zforce $ORIGINAL # Restore the original when proxy removed
		rm zforce-proxy
	fi
fi

Basically, all I had to do was add those rules to /etc/udev/rules.d, add the script to /etc/udev and then place the dummy kernel module (lazy, I just put it on root) and then patch the rcS script to load it for a test.

patch for rcS.

# Add these lines to /etc/init.d/rcS right after DBUS_SESSION_BUS_ADDRESS=
insmod /proxyevent.ko
hostname YourNameForKobo
udevadm trigger
udevadm settle
cat /dev/input/zforce > /dev/input/zforce-proxy &
/mnt/onboard/autorun.sh &
sleep 2

The dummy driver and udev script works really well on my AuraHD ; I just have my keyboard driver kill the 'cat' function, and then replace it with a keyboard filter script (not shown) to switch back and forth between a framebuffer terminal program, and the kobo reader.

But for some strange reason, the Kobo GLO's the system appears to crash andomly when not being used (idle) for a minute or more. I thought the GLO and AuraHD were supposed to use the same firmware, so I'm not sure what's wrong.

42d520248dba93f5ae5299e7604e5260 proxymodule.tar.gz
de6bd9fc931a1ff003a8dacd1d02f3d3 udevzforce.tar.gz

[davidfor]

Originally Posted by NiLuJe

IIRC, syslog doesn't log to a file on the Kobos, it only keeps track of the n'th latest events in memory, which you can access via logread.

Add the following to the config file:

[DeveloperSettings]
EnableDebugServices=true

This writes to a file in the .kobo directory with name including the timestamp, such as "syslog-2015.06.10-20.41.21". It gets written when you restart or connect to the PC. Plus the last two or three firmware versions have a button to save the log on the "Device information" page of the settings.

If anyone you add this, watch the battery. This also enables telnet and if WiFi is active, it doesn't time out and the auto sleep/power off doesn't kick in. A sleep cover still works.

[NiLuJe]

@davidfor: Neat info, thanks!

[PeterT]

Originally Posted by davidfor

Add the following to the config file:

[DeveloperSettings]
EnableDebugServices=true

This writes to a file in the .kobo directory with name including the timestamp, such as "syslog-2015.06.10-20.41.21". It gets written when you restart or connect to the PC. Plus the last two or three firmware versions have a button to save the log on the "Device information" page of the settings.

If anyone you add this, watch the battery. This also enables telnet and if WiFi is active, it doesn't time out and the auto sleep/power off doesn't kick in. A sleep cover still works.

Sheesh; finally sharing the telnet feature eh?

[davidfor]

Originally Posted by PeterT

Sheesh; finally sharing the telnet feature eh?

Didn't you know about that? And to be honest, I've been wondering why no-one had ever mentioned this. The options aren't hard to find in libnickel.

I suppose I should point out there four other developer settings:

• EnablePacketDump
• IgnoreSSLErrors
• GeoClientIP
• EnableInspector

I haven't tried "GeoClientIP", but the others haven't done anything that I can find.

[PeterT]

I've been using the telnet for a long long time

[fastrobot]

Originally Posted by PeterT

I've been using the telnet for a long long time

I ended up writing a telnet replacement program based on one I had for my Sony reader because my attempts to follow instructions for enabling Telnet for Kobo on various threads I found on MRforums didn't work for me; eg: busybox for the AuraHD didn't have the telnet server or ftp compiled in when I first got it.

I take it, this telnet server you're talking about is part of nickel -- or am I wrong, and some releases of the firmware support Telnet as part of busybox, and sometimes doesn't ? Or does the ability to enable Telnet depend on which kind of Kobo you own ?

This kind of inconsistency is irritating.... as it's very time consuming to sort out....

I had to give my kids the kobo Glo's I had to experiment with to do their book reading with this week, and that's with the system still locking up every so often if they don't press a key. eg: I ran out of time to find and fix the bug.

On the other hand, my AuraHD has been running the switch mode framebuffer without problems for two days now, no lockups.

So -- I'm buying a third KoboGlo as a spare to do experiments on. But It'll likely be a week before I get it in the mail...


Question: Do you know if the Glo's ARM processor has hardware floating point, or is it somehow different than the AuraHD's processor which might trip up a compiler?

I was wondering if that was different between the Glo and the AuraHD, and might be causing the failures I'm seeing because Kobo's developer tool package has two different gcc cross compilers; and I installed both, they are executed with;

arm-linux-gnueabihf-gcc and arm-none-linux-gnueabi-gcc
for linaro and code sourcery compiler packages.

The one with the 'hf' abi, I think, is a compiler supporting hardware floating point instructions, and that compiler is set up to compile user space applications that link to the standard c libraries. eg: glibc. etc.

But -- The other compiler, is the one I assumed the linux kernel was compiled with; since the linux kernel can do floating point emulation and often doesn't need the standard C libraries.

Am I mistaken ?

Should I have compiled the Kobo Glo's kernel module using the Code sourcery version of the complier instead of the linaro version ?

[fastrobot]

Originally Posted by davidfor

Add the following to the config file:

[DeveloperSettings]
EnableDebugServices=true

This writes to a file in the .kobo directory with name including the timestamp, such as "syslog-2015.06.10-20.41.21". It gets written when you restart or connect to the PC. Plus the last two or three firmware versions have a button to save the log on the "Device information" page of the settings.

If anyone you add this, watch the battery. This also enables telnet and if WiFi is active, it doesn't time out and the auto sleep/power off doesn't kick in. A sleep cover still works.

Cool. I'll have to try this later.

Another Question....

I see in the KOBO-labs git hub repository, that the cross compiler binaries are also compiled with remote gdb capability. eg: arm-linux-gnueabihf-gdb and arm-none-linux-gnueabi-gdb both exist that will run on an x86PC but should debug an arm binary remotely.

So, these ought to be ready to talk to a gdb proxy on the Kobo, and allow me to do debugging remotely (at least on programs I compile myself with -ggdb3).

But -- i don't see a gdbstub, or gdbproxy binary compiled for Arm in the repository.
Do you know where the GDB proxy binary might be found, or how it could be enabled, so I can do remote debugging ?

[davidfor]

Originally Posted by fastrobot

I take it, this telnet server you're talking about is part of nickel -- or am I wrong, and some releases of the firmware support Telnet as part of busybox, and sometimes doesn't ? Or does the ability to enable Telnet depend on which kind of Kobo you own ?

This kind of inconsistency is irritating.... as it's very time consuming to sort out....

I have no idea where it is. The options are in libnickel and I assume it interprets them. Whether the telnet implementation is part of nickel or it simply enables something somewhere else, I don't know and have only a passing interest in finding out. When I'm curious about something, it works.

What inconsistency? As I keep saying, and no one seems to believe me, the firmware is identical between the devices. I probably need to change this "virtually identical" as there are some drivers that are different in the Glo HD update package. The other differences are fonts and the pictures used in the slideshow. And yes the kernel is different and that should account for the rest of the hardware differences but is probably built from a common source. The thing is that what most of us see when using the device is identical running taking the hardware differences into account. And everything running behind the scenes is the same.

And specifically, the developer options and telnet work on all the Kobo devices I have. The options has been in the firmware for some time. Exactly what they do and what options there are have changed (there was a "LogTouches" option at one point), but since they were added, they work in exactly the same way on all the devices running the same firmware levels. The same goes for all other options that aren't hardware dependent (no light, no function and only the Aura supports mult-touch).

For the rest of your question, sorry I can't help. I have little experience and knowledge in this area, and to be honest, very little interest.