Onyx Boox as monitor

[sogaiu]

Originally Posted by mdp

Powering the tablet is not a problem, you can do it through the same PC providing the desktop. (And with "Normal" EPD mode, and if you do not have those glitches that update part of the screen every heartbit, you may not even need to power it during operation: the device would discharge much slower.)

FWIW, to test the log-monitoring for detection of some position info, I had connected the Max 2 to a PC's USB port, and when I stopped testing just now, the battery was down to 10% -- when I had started testing it was close to 100%. (I don't remember whether I was using Normal or A2.)

The Monitor app also mentioned that power might be a problem when I started it up. When I have the Max 2 connected to a USB power adapter, I don't recall getting such messages.

[mdp]

Originally Posted by sogaiu

I didn't do the clicking part

Tested, it works!
One note: I think `logcat` should be already opened with the filtering (similar to the above I posted), so you process something already cleaned up - which I presume should be faster.

The good idea about python is that it will work everywhere - linux, win, mac - ; it should be seen if this whole idea will be fast enough.

For doing more native code, posting mouse events should be trivial in Win (the functions are there in the Win32 API), but I have not understood yet the low level trick in Linux.

Anyway, as I wrote the Monitor code should probably come to include the tap-down (now it only gets "tap up"), so that also drag-and-drop could work...

[mdp]

Originally Posted by sogaiu

FWIW, to test the log-monitoring for detection of some position info, I had connected the Max 2 to a PC's USB port, and when I stopped testing just now, the battery was down to 10% -- when I had started testing it was close to 100%. (I don't remember whether I was using Normal or A2.)

The Monitor app also mentioned that power might be a problem when I started it up. When I have the Max 2 connected to a USB power adapter, I don't recall getting such messages.

The A2 rendering implemented with the Floyd-Steinberg is beautiful, but not optimized to be conservative in dynamic contexts, and it moves more dots than a Farbrausch demoscene party Produkkt...

[mdp]

So since we understood that there may be a simple method that allows to use the HDMI based "Monitor" software from Onyx also as a touchpad - in other words, the Max2 (etc.) becomes a touchscreen monitor - I further modified the (modifications on the) original code to allow the magic with the tricks we can afford here.

Now, with this version:
-- tapping on the NW corner ("corner" = within 100 dots) opens the original menu. Tapping on the main region of the screen does not open the menu anymore, because you are supposed to able to use it as a touchscreen;
-- tapping on the NE corner requests a full screen refresh;
-- tapping on the SW corner changes EPD mode to Normal / high quality;
-- tapping on the SE corner changes EPD mode to A2 / fast mode;
-- long tap does not trigger any more the "Exit?" menu, because you should be allowed to drag the mouse pointer around.

Upon touch, the application logs the taps the following way:

03-07 22:32:02.247: I/OnyxMonitEv(3434): T:0|1643,716
03-07 22:32:02.403: I/OnyxMonitEv(3434): T:1|1593,733
03-07 22:32:03.520: I/OnyxMonitEv(3434): T:0|623,849
03-07 22:32:03.960: I/OnyxMonitEv(3434): T:1|1202,539
03-07 22:32:05.520: I/OnyxMonitEv(3434): T:0|1694,814
03-07 22:32:05.637: I/OnyxMonitEv(3434): T:1|1694,814

and these logs could be used to reflect with mouse actions on the source desktop.

Sogaiu has already written a python script that does the job, I will soon modify it to match with this "more precise" Monitor version.

For the technically inclined, I also attached the main "modification" (outside this files the interventions are very scarce).

[mdp]

@Sogaiu:
almost there,
the "popen" now must be
process = subprocess.Popen(['adb', 'logcat', 'OnyxMonitEv:I *:S'], stdout=subprocess.PIPE)
and the regular expression
line_re = re.compile("^(\d+)-(\d+) (\d+)\d+)\d+)\.(\d+).*?OnyxMonitEv: T?P<e>\d+)\|(?P<x>\d+),(?P<y>\d+)")

One scolio, the coordinates must be modified according to screen resolution (e.g. 2104x1560) and display resolution (presumably 2200x1650) - such as
mouse_x = tap_x*2104/2200
I cannot finish this evening...

[mdp]

...and it works like a charm!

I cannot finish the script, but already like it is (dirty, with hardcoded values etc.) it plain works! Check the video out! :champagne:

The core, for quick replication of the results:

	while not stdout_reader.eof():
		while not stdout_queue.empty():
			line = stdout_queue.get()
			print(line)
			m = line_re.match(line)
			if m:
				e = int(m.group("e"))
				x = int(m.group("x"))*2104/2200
				y = int(m.group("y"))*1560/1650
				if(e==0):
					mouse.press(x,y)
				elif(e==1):
					mouse.release(x,y)

Another thing to do: first a loop to exhaust the queue (to eliminate the old commands in the log without them having effect)...

The Onyx Monitor software fully improved to a touchscreen. I am impressed myself...

[sogaiu]

Nice!

I watched the video, but also made similar modifications and verified it works

Some points I noticed:

-The script is very CPU intensive. Not sure why yet.
-I guess this setup works for mirrored screens, but perhaps some modifications are necessary for other modes of operation like "extended".
-Not noticing the same kind of power drain as the last round of testing.

[mdp]

Originally Posted by mdp

Another thing to do: first a loop to exhaust the queue (to eliminate the old commands in the log without them having effect)...

Just noting: the above is not necessary (and not exception proof), instead I would do a
logcat -T 1
and just skip processing that first line.

Any request for a demo?

[sogaiu]

Originally Posted by sogaiu

-The script is very CPU intensive. Not sure why yet.

I think I addressed this by adding time.sleep(0.01) to the outermost while loop. This calmed things down quite a bit. Other values may be more appropriate.

[sogaiu]

Also working on skipping lines that are before the time that the script started executing. Going to assume that the device's time and the host's time are close. Looks like adb logcat output doesn't contain year info...

[mdp]

Originally Posted by sogaiu

Also working on skipping lines that are before the time that the script started executing. Going to assume that the device's time and the host's time are close. Looks like adb logcat output doesn't contain year info...

You don't need that, just use that "-T1" as I mentioned above: you will only dump one past line, instead of the whole history. The "Popen" string becomes:
process = subprocess.Popen(['adb', 'logcat', '-T1', 'OnyxMonitEv:I *:S'], stdout=subprocess.PIPE)

EDIT: should you wish to accept your mission, the script should, at the begnning, check the screen resolution. Because those hardcoded values in

				x = int(m.group("x"))*2104/2200
				y = int(m.group("y"))*1560/1650

Should go into variables.
The other two values, the tablet resolution, could be taken with another adb command (maybe an `adb shell getprop [...]`

EDIT: more exactly,
`adb shell getprop ro.product.board` returns 'Max2'.
You can use a switch on that output to impose that tablet_res_w = 2200 and tablet_res_h = 1650.
With the Note, which will get another value as "ro.product.board", the conditional will acquire another branch towards values tablet_res_w = 1872 and tablet_res_h = 1404.

[mdp]

Meanwhile, I took a video. I mean, so amazing that it has to be more properly demonstrated: the Max2 in an HDMI monitor session, acting as a touchscreen.

https://youtu.be/apzWhVjpS6o



The current (unimaginative) description:

Onyx Boox Max2 used as monitor through the HDMI port, with customizations in the Monitor software that also allow to use it as a touch-screen.

[sogaiu]

Originally Posted by mdp

You don't need that, just use that "-T1" as I mentioned above

Ah, I see. Thanks for the persistence

On a side note, it seems silly that there's both a -T <count> and a -T '<time>'

should you wish to accept your mission, the script should, at the begnning, check the screen resolution. Because those hardcoded values in

				x = int(m.group("x"))*2104/2200
				y = int(m.group("y"))*1560/1650

Should go into variables.

The local version does this. Any thoughts on what to do for non-mirroring arrangements?

The other two values, the tablet resolution, could be taken with another adb command (maybe an `adb shell getprop [...]`

EDIT: more exactly,
`adb shell getprop ro.product.board` returns 'Max2'.
You can use a switch on that output to impose that tablet_res_w = 2200 and tablet_res_h = 1650.

Thanks for the pointers -- intend to try this out.

[sogaiu]

Originally Posted by mdp

Meanwhile, I took a video.

May be the folks with interest in using the Max 2 with the Raspberry Pi will be interested too.

I think I saw the "touching the bottom left corner" functionality working still -- is that correct? If so, is that distinguishable from a click / touch intended for the desktop? I guess there is an answer in the .smali


P.S. The double-clicking looked slightly tricky. How was that for you?

[mdp]

Originally Posted by sogaiu

Any thoughts on what to do for non-mirroring arrangements?

I think the (mouse) coordinates get an offset:
the most frequent case could be:
mouse_x = main_screen_width + second_screen_x
mouse_y = second_screen_y
meaning the extension is eastward, but it could be anything. A setting must be read to determine it.