Clara (Pre-FBInk) rmkit on my Clara HD

[Quoth]

Originally Posted by elinkser

But this is open source, so I imagine if Xournal collects the strokes, some other layer could do the conversion to characters?

Conversion to real computer text, shape recognition, math equation recognition and check box recognition is what the Advanced notebook on Sage & Elipsa does, as does Nebo on Android (some models) and newer iPads (Apple Pencil). That's really really hard stuff. OCR software (printed text images) has open source options but not handwriting conversion.

Xournal is pointless if you have Nebo or similar.

[elinkser]

***

The landscape does look pretty bleak so not arguing with you but, for the record:


[WIP] Lua Plugin for Handwriting recognition #2176
https://github.com/xournalpp/xournalpp/pull/2176
There is a link to this:
Lipi Toolkit
https://lipitk.sourceforge.net/lipi-toolkit.htm
https://sourceforge.net/projects/lip...pi_toolkit4.0/

$ cd rmkit-build/

$ tar zxvf lipi-toolkit4.0.0-src-linux.tar.gz lipi-toolkit4.0.0-src-linux/

$ cd lipi-toolkit4.0.0-src-linux/

$ nano -l src/build-x64.sh

 80 #       if [ $os != "Ubuntu" ]
 81 #       then
 82 #               echo "\nERROR: lipi-core-toolkit can be built only on Ubuntu Linux\n"
 83 #               exit 1
 84 #       fi
...
 88                 find /lib/x86_64-linux-gnu -print | grep "$filename\." 1>/dev/null 2>&1

$ sh src/build-x64.sh -b
Building lipitk objects and binaries.....OK

$ LIPI_ROOT=~/rmkit-build/lipi-toolkit4.0.0-src-linux src/apps/samples/shaperectst/linux-x64/shaperectst SHAPEREC_NUMERALS projects/demonumerals/data/1/1_0.txt

Input Logical project name = SHAPEREC_NUMERALS

Input ink file for recognition = projects/demonumerals/data/1/1_0.txt

Recognition Results

Choice[0] Recognized Shapeid = 1 Confidence = 0.549951

Choice[1] Recognized Shapeid = 2 Confidence = 0.126473


Over a decade old and still ticking!

***

jersou/mouse-actions
https://github.com/jersou/mouse-actions

$ cd rmkit-build/

$ wget https://github.com/jersou/mouse-acti...heads/main.zip

$ mv main.zip mouse-actions-main.zip

$ unzip mouse-actions-main.zip

$ cd mouse-actions-main/

$ cargo build --release

Ran out of cell data, so had to abort, LOL

[elinkser]

Originally Posted by NiLuJe

Currently wouldn't work on a Sage/Elipsa, that's another SoC & display driver entirely.

Darn - I hope it gets supported, if Kobo is moving to that SoC.
I was actually considering the Sage as an upgrade, but now my Clara HD is looking pretty good!

[NiLuJe]

Originally Posted by elinkser

if Kobo is moving to that SoC.

They're (hopefully) not, they're (most likely) moving to the Elipsa 2E's SoC (MTK), like lab126.

[elinkser]

Originally Posted by NiLuJe

They're (hopefully) not, they're (most likely) moving to the Elipsa 2E's SoC (MTK), like lab126.

Hmm, so we at least had one testimony of rmkit working on the Elipsa 2E:
https://www.mobileread.com/forums/sh...46&postcount=3

And looking at the rest of the Kobo lineup, I see mostly Freescale (other than Sage/Elipsa):
https://en.m.wikipedia.org/wiki/Kobo_eReader

So can we infer that the other Kobos should support rmkit?


But there's something {among many other things) that still troubles me.
The Pocketbook lineup appears to be largely Allwinner, like the dreaded Sage/Elipsa:
https://en.m.wikipedia.org/wiki/Pock..._International

Yet by some feate of Magicke there has appeared fbink supporton the Pocketbook:
https://github.com/NiLuJe/FBInk

Some of the conjurations have even been revealed:
https://github.com/NiLuJe/FBInk/pull/64


So is there no way to use a similar incantation for rmkit??

[elinkser]

POSSIBLE SECURITY ISSUES OF RMKIT


Since rmkit makes it easier to quickly develop cross-platform apps, some might roll their eyes and say, "Great, more junk apps".


Security-conscious people should also be concerned that a proliferation of low quality apps makes a nice target for malware.

-C++ requires effort and skill to be done correctly, and therefore securely.

-Kobo Nickel is designed for reading, not networking. It lacks up-to-date kernel, fine-grained permissions in the user partition, and kernel support for firewalls, among other issues.


How can this problem be addressed?
Kobo is even less equipped to deal with this than old Windows was.


Some random ill-thought out musings:

- Use InkBox instead of Nickel

- Only allow signed apps to launch in rmkit

- Run an app scanner over apps in the rmkit repository.

- Run a user-space firewall in rmkit (all i/o accesses only allowed through "secure" api)

- Require users who want to run Simple framework to self-sign any app they want to give run permission to.



Who would do all this work?
Not me - I'm not a dev, and definitely not a C++ dev.
But I want to enjoy my Kobo in a secure way.

[NiLuJe]

Originally Posted by elinkser

But there's something {among many other things) that still troubles me.
The Pocketbook lineup appears to be largely Allwinner, like the dreaded Sage/Elipsa:

No clue how that currently stands, but, when they moved to AW (on much older SoCs than the one used by the Sage/Elipsa, FWIW), they wrote a driver shim that (badly) translated the public mxcfb API to the (crappy) v1 AW display API.

We tried to live with it for a while in KOReader, but it required a gigantic and brittle pile of hacks, so we simply caved in and now go through the native middleware (InkView).

FBInk doesn't (as much as possible, we still use it for device identification, although that can be disabled, because bringing in InkView introduces brand new ways to break stuff in fun and interesting ways), and still includes the bare minimum of said pile of hacks.

TL;DR: It works. Not well, but it "works". In any case, definitely *not* an example to follow.

[Szybet]

Originally Posted by elinkser

POSSIBLE SECURITY ISSUES OF RMKIT


Since rmkit makes it easier to quickly develop cross-platform apps, some might roll their eyes and say, "Great, more junk apps".


Security-conscious people should also be concerned that a proliferation of low quality apps makes a nice target for malware.

-C++ requires effort and skill to be done correctly, and therefore securely.

-Kobo Nickel is designed for reading, not networking. It lacks up-to-date kernel, fine-grained permissions in the user partition, and kernel support for firewalls, among other issues.


How can this problem be addressed?
Kobo is even less equipped to deal with this than old Windows was.


Some random ill-thought out musings:

- Use InkBox instead of Nickel

- Only allow signed apps to launch in rmkit

- Run an app scanner over apps in the rmkit repository.

- Run a user-space firewall in rmkit (all i/o accesses only allowed through "secure" api)

- Require users who want to run Simple framework to self-sign any app they want to give run permission to.



Who would do all this work?
Not me - I'm not a dev, and definitely not a C++ dev.
But I want to enjoy my Kobo in a secure way.

I would screem if you didn't mentioned InkBox ;p
Running rmkit as a InkBox user apps makes it extremely sandboxed, but it wouldn't protect against all ( Only some, we limit access to /dev and /sys ) kernel level exploits.
And you worry about it too much. Who cares about the x users using rmkit? You would need to special prepare malware for what?

[elinkser]

Originally Posted by Szybet

And you worry about it too much.

I prefer the term "security mindset".

[raisjn]

So can we infer that the other Kobos should support rmkit?

This is on a case by case basis, I typically use NiLuJe's fbink to look up what support is necessary for a device. the advice I had previously received from NiLuJe is to get the Clara working because that is similar to the most other Kobo devices. In terms of supporting that device listed - yes, it seems possible, but we would end up doing a lot of work. My ideal is to somehow use fbink as the backend for rmkit's notion of a framebuffer. then we can say anywhere fbink works, rmkit is supported (for drawing to screen). the main issue after that is aligning the input coordinates with the display coordinates

Security-conscious people should also be concerned that a proliferation of low quality apps makes a nice target for malware.

this is a fair point. I think that the major problem is that all readers are running as root and we haven't thought about what it takes to run an app as non root. I know that Oxide (for rM) does do sandboxing with a chroot. overall, this could be something to investigate for remux. for now, we have to trust that the source code does what it says and that the community is not being malicious.

i'm not a security person, so i don't know how they grade security issues, but i think there are two major portions: opportunity size and risk size. the opportunity is low (not that many users: maybe dozens or low hundreds, i have no way of knowing) and the risk is also low (people do not keep sensitive personal information on their readers. the major risk is having your data deleted on you or someone install spyware to report on your activity). probably we mostly need to watch out for people who get some fun out of harming other people's daily life (a griefer)

[elinkser]

@raisjn

I appreciate your consideration of my concerns.

I'm familiar enough about computer security to avoid using a cool network app to connect from my insecure kobo to my phone I use for banking, but I don't necessrily expect regular users to be.
They may have an expectation that they are being taken care of.


On a funner note, I finally got around to trying the remarkable port of Simon Tatham's Portable Puzzle Collection!
(I first tried the binary, then recompiled it for my Clara HD - attached).
mv puzzles.bin puzzles.xz
xz -d puzzles.xz


First tried the binary:

mrichards42/remarkable_puzzles
https://github.com/mrichards42/remarkable_puzzles

Simon Tatham's Portable Puzzle Collection for the reMarkable tablet.

$ mkdir mypuzzles

$ cd mypuzzles/

$ wget https://github.com/mrichards42/remar...puzzles.tar.gz

$ tar zxvf puzzles.tar.gz

On the kobo create etc/puzzles/ folder in /mnt/onboard/.adds/rmkit/.

Copy the config/, help/, icons/ folders to the /mnt/onboard/.adds/rmkit/etc/puzzles/ folder.

Copy the puzzles binary to the /mnt/onboard/.adds/rmkit/bin/apps/ folder.

/mnt/onboard/.adds/rmkit/bin/apps/puzzles: /lib/libm.so.6: version `GLIBC_2.27' not found (required by /mnt/onboard/.adds/rmkit/bin/apps/puzzles)


OK let's try to build:

$ git clone --recurse-submodules https://github.com/mrichards42/remarkable_puzzles.git

$ cd remarkable_puzzles/

$ source ~/koxtoolchain/refs/x-compile.sh kobo env bare

$ nano -l defs.mk

...
 11 export BUILD ?= release
...
 26 else ifeq ($(ARCH),kobo)
 27         CXX = arm-kobo-linux-gnueabihf-g++
 28         CC  = arm-kobo-linux-gnueabihf-gcc
 29         STB = stb.arm.o

$ nano -l vendor/rmkit/src/common.make

  3 CXX_BIN?=arm-kobo-linux-gnueabihf-g++
  4 CC_BIN?=arm-kobo-linux-gnueabihf-gcc
  5 STRIP_BIN?=arm-kobo-linux-gnueabihf-strip

$ nano -l vendor/rmkit/src/rmkit/input/events.cpy

241 //        case ABS_MT_SLOT:
242 //          slot = data.value;
243 //          break
...
282 //        case ABS_MT_DISTANCE:
283 //          slots[slot].distance = self.distance = data.value
284 //          if self.distance > 0:
285 //            self.left = 0
286 //          break

$ nano -l vendor/rmkit/src/rmkit/input/input.cpy

 69 //          if ev_data[i].code == SYN_DROPPED:
 70 //            syn_dropped = true
 71 //            event.handle_drop(fd)
 72 //            continue

$ nano -l vendor/rmkit/src/remux/launcher.cpy

 31 #define TOUCH_FLOOD_EVENT ABS_DISTANCE

$ ARCH=kobo make puzzles

$ cp build/release/puzzles .

$ arm-kobo-linux-gnueabihf-strip -s puzzles
$ ls -l puzzles
-rwxr-xr-x 1 1892712 puzzles

$ zip puzzles.zip puzzles
$ ls -l puzzles.zip
-rw-r--r-- 1 1119921 puzzles.zip

$ xz puzzles
$ ls -l puzzles.xz
-rwxr-xr-x 1 919152 puzzles.xz

On the kobo create etc/puzzles/ folder in /mnt/onboard/.adds/rmkit/.

Copy the config/, help/, icons/ folders to the /mnt/onboard/.adds/rmkit/etc/puzzles/ folder.

Copy the puzzles binary to the /mnt/onboard/.adds/rmkit/bin/apps/ folder.
mv puzzles.bin puzzles.xz
xz -d puzzles.xz

Works great, puzzle lovers rejoice!

[Clouds]

Would it be possible to get rmkit working on an Aura One?
I installed rmkit, which according to ps | grep remux is running, but doesn't appear to do anything beyond that.

As per https://rmkit.dev/rmkit-comes-to-kobo/ by @raisjn I guess support needs to be added first (if possible at all)?

[elinkser]

Originally Posted by Clouds

Would it be possible to get rmkit working on an Aura One?
I installed rmkit, which according to ps | grep remux is running, but doesn't appear to do anything beyond that.

As per https://rmkit.dev/rmkit-comes-to-kobo/ by @raisjn I guess support needs to be added first (if possible at all)?

Supported devices (Libra H20 and Clara HD) use a different chipset than Aura One.
https://en.m.wikipedia.org/wiki/Kobo_eReader

The developer says in your link:
Currently, the Libra H20 and Clara are supported, but if you have any requests, please get in touch and I’ll look into enabling more devices

If there is enough interest, hopefully Raisjn will port to FBInk as a general solution, though device touch input handling is also needed.

Here is what I used to debug in Post#7:

When I run from SSH on PC:

# ps | grep remux
1514 root 0:00 sh /opt/bin/remux.sh
1516 root 0:00 /opt/bin/apps/remux
1648 root 0:00 grep remux

# kill -KILL 1514
# kill -KILL 1516

# sh /opt/bin/remux.sh

Then swipe up to start remux, and select e.g the drawing_demo app:
DISPLAYED LAUNCHER FOR 2200 MS
LAUNCHING APP drawing_demo drawing_demo
POWER MANAGEMENT: 1
RESETTING MT GESTURES
CHECKING PROCESS drawing_demo 0
RENDERING drawing_demo
DECOMP TOOK 0.00358233
RUNNING ON CLARA HD
SETTING SCREEN DEPTH 32
W: 1072 H: 1448 S: 1088
XRES 1072 YRES 1448 BPP 32 GRAYSCALE 0
OPENING /dev/input/event0 AS BUTTONS
OPENING /dev/input/event1 AS TOUCH
ERROR OPENING INPUT DEVICE /dev/input/event2
ERROR OPENING INPUT DEVICE /dev/input/event3
ERROR OPENING INPUT DEVICE /dev/input/event4
on_mouse_move debug, ev= 262
on_mouse_move debug, prevx= 262
on_mouse_move debug, ev= 262
on_mouse_move debug, prevx= 262
on_mouse_move debug, ev= 262
on_mouse_move debug, ev= 263
on_mouse_move debug, prevx= 263


*Note that you have to swipe up along the edge of the display to bring the up the rmkit menu. (I would try swiping all directions in your case since the touch support is not specified for your device. If you are lucky, you will get messages like above.

[Clouds]

Thanks for your reply.

When I do what you did for debugging, # sh /opt/bin/remux.sh results in an endless loop of

FATAL: kernel too old
Aborted

I take that to mean rmkit doesn't currently work on KA1.
Whether that is because of the different chipset or simply because the device isn't added as supported yet (as per "Individual devices have to be enabled so as to not cause issues, so seriously: please get in touch!"), I don't know.

Hopefully Raisjn can shed some light on that

[elinkser]

Yes, letting Raisjn know would be good for supporting older devices.

I did try building the remux bin with an older kernel - you can try it if you want.