Clara REMARKABLE RMKIT TOOLKIT AND SIMPLE -DEVELOP CROSSPLATFORM WITHOUT COMPILING

[Quoth]

Originally Posted by elinkser

But this is open source, so I imagine if Xournal collects the strokes, some other layer could do the conversion to characters?

Conversion to real computer text, shape recognition, math equation recognition and check box recognition is what the Advanced notebook on Sage & Elipsa does, as does Nebo on Android (some models) and newer iPads (Apple Pencil). That's really really hard stuff. OCR software (printed text images) has open source options but not handwriting conversion.

Xournal is pointless if you have Nebo or similar.

[elinkser]

***

The landscape does look pretty bleak so not arguing with you but, for the record:


[WIP] Lua Plugin for Handwriting recognition #2176
https://github.com/xournalpp/xournalpp/pull/2176
There is a link to this:
Lipi Toolkit
https://lipitk.sourceforge.net/lipi-toolkit.htm
https://sourceforge.net/projects/lip...pi_toolkit4.0/

$ cd rmkit-build/

$ tar zxvf lipi-toolkit4.0.0-src-linux.tar.gz lipi-toolkit4.0.0-src-linux/

$ cd lipi-toolkit4.0.0-src-linux/

$ nano -l src/build-x64.sh

 80 #       if [ $os != "Ubuntu" ]
 81 #       then
 82 #               echo "\nERROR: lipi-core-toolkit can be built only on Ubuntu Linux\n"
 83 #               exit 1
 84 #       fi
...
 88                 find /lib/x86_64-linux-gnu -print | grep "$filename\." 1>/dev/null 2>&1

$ sh src/build-x64.sh -b
Building lipitk objects and binaries.....OK

$ LIPI_ROOT=~/rmkit-build/lipi-toolkit4.0.0-src-linux src/apps/samples/shaperectst/linux-x64/shaperectst SHAPEREC_NUMERALS projects/demonumerals/data/1/1_0.txt

Input Logical project name = SHAPEREC_NUMERALS

Input ink file for recognition = projects/demonumerals/data/1/1_0.txt

Recognition Results

Choice[0] Recognized Shapeid = 1 Confidence = 0.549951

Choice[1] Recognized Shapeid = 2 Confidence = 0.126473


Over a decade old and still ticking!

***

jersou/mouse-actions
https://github.com/jersou/mouse-actions

$ cd rmkit-build/

$ wget https://github.com/jersou/mouse-acti...heads/main.zip

$ mv main.zip mouse-actions-main.zip

$ unzip mouse-actions-main.zip

$ cd mouse-actions-main/

$ cargo build --release

Ran out of cell data, so had to abort, LOL

[elinkser]

Originally Posted by NiLuJe

Currently wouldn't work on a Sage/Elipsa, that's another SoC & display driver entirely.

Darn - I hope it gets supported, if Kobo is moving to that SoC.
I was actually considering the Sage as an upgrade, but now my Clara HD is looking pretty good!

[NiLuJe]

Originally Posted by elinkser

if Kobo is moving to that SoC.

They're (hopefully) not, they're (most likely) moving to the Elipsa 2E's SoC (MTK), like lab126.

[elinkser]

Originally Posted by NiLuJe

They're (hopefully) not, they're (most likely) moving to the Elipsa 2E's SoC (MTK), like lab126.

Hmm, so we at least had one testimony of rmkit working on the Elipsa 2E:
https://www.mobileread.com/forums/sh...46&postcount=3

And looking at the rest of the Kobo lineup, I see mostly Freescale (other than Sage/Elipsa):
https://en.m.wikipedia.org/wiki/Kobo_eReader

So can we infer that the other Kobos should support rmkit?


But there's something {among many other things) that still troubles me.
The Pocketbook lineup appears to be largely Allwinner, like the dreaded Sage/Elipsa:
https://en.m.wikipedia.org/wiki/Pock..._International

Yet by some feate of Magicke there has appeared fbink supporton the Pocketbook:
https://github.com/NiLuJe/FBInk

Some of the conjurations have even been revealed:
https://github.com/NiLuJe/FBInk/pull/64


So is there no way to use a similar incantation for rmkit??

[elinkser]

POSSIBLE SECURITY ISSUES OF RMKIT


Since rmkit makes it easier to quickly develop cross-platform apps, some might roll their eyes and say, "Great, more junk apps".


Security-conscious people should also be concerned that a proliferation of low quality apps makes a nice target for malware.

-C++ requires effort and skill to be done correctly, and therefore securely.

-Kobo Nickel is designed for reading, not networking. It lacks up-to-date kernel, fine-grained permissions in the user partition, and kernel support for firewalls, among other issues.


How can this problem be addressed?
Kobo is even less equipped to deal with this than old Windows was.


Some random ill-thought out musings:

- Use InkBox instead of Nickel

- Only allow signed apps to launch in rmkit

- Run an app scanner over apps in the rmkit repository.

- Run a user-space firewall in rmkit (all i/o accesses only allowed through "secure" api)

- Require users who want to run Simple framework to self-sign any app they want to give run permission to.



Who would do all this work?
Not me - I'm not a dev, and definitely not a C++ dev.
But I want to enjoy my Kobo in a secure way.

[NiLuJe]

Originally Posted by elinkser

But there's something {among many other things) that still troubles me.
The Pocketbook lineup appears to be largely Allwinner, like the dreaded Sage/Elipsa:

No clue how that currently stands, but, when they moved to AW (on much older SoCs than the one used by the Sage/Elipsa, FWIW), they wrote a driver shim that (badly) translated the public mxcfb API to the (crappy) v1 AW display API.

We tried to live with it for a while in KOReader, but it required a gigantic and brittle pile of hacks, so we simply caved in and now go through the native middleware (InkView).

FBInk doesn't (as much as possible, we still use it for device identification, although that can be disabled, because bringing in InkView introduces brand new ways to break stuff in fun and interesting ways), and still includes the bare minimum of said pile of hacks.

TL;DR: It works. Not well, but it "works". In any case, definitely *not* an example to follow.

[Szybet]

Originally Posted by elinkser

POSSIBLE SECURITY ISSUES OF RMKIT


Since rmkit makes it easier to quickly develop cross-platform apps, some might roll their eyes and say, "Great, more junk apps".


Security-conscious people should also be concerned that a proliferation of low quality apps makes a nice target for malware.

-C++ requires effort and skill to be done correctly, and therefore securely.

-Kobo Nickel is designed for reading, not networking. It lacks up-to-date kernel, fine-grained permissions in the user partition, and kernel support for firewalls, among other issues.


How can this problem be addressed?
Kobo is even less equipped to deal with this than old Windows was.


Some random ill-thought out musings:

- Use InkBox instead of Nickel

- Only allow signed apps to launch in rmkit

- Run an app scanner over apps in the rmkit repository.

- Run a user-space firewall in rmkit (all i/o accesses only allowed through "secure" api)

- Require users who want to run Simple framework to self-sign any app they want to give run permission to.



Who would do all this work?
Not me - I'm not a dev, and definitely not a C++ dev.
But I want to enjoy my Kobo in a secure way.

I would screem if you didn't mentioned InkBox ;p
Running rmkit as a InkBox user apps makes it extremely sandboxed, but it wouldn't protect against all ( Only some, we limit access to /dev and /sys ) kernel level exploits.
And you worry about it too much. Who cares about the x users using rmkit? You would need to special prepare malware for what?

[elinkser]

Originally Posted by Szybet

And you worry about it too much.

I prefer the term "security mindset".

[raisjn]

So can we infer that the other Kobos should support rmkit?

This is on a case by case basis, I typically use NiLuJe's fbink to look up what support is necessary for a device. the advice I had previously received from NiLuJe is to get the Clara working because that is similar to the most other Kobo devices. In terms of supporting that device listed - yes, it seems possible, but we would end up doing a lot of work. My ideal is to somehow use fbink as the backend for rmkit's notion of a framebuffer. then we can say anywhere fbink works, rmkit is supported (for drawing to screen). the main issue after that is aligning the input coordinates with the display coordinates

Security-conscious people should also be concerned that a proliferation of low quality apps makes a nice target for malware.

this is a fair point. I think that the major problem is that all readers are running as root and we haven't thought about what it takes to run an app as non root. I know that Oxide (for rM) does do sandboxing with a chroot. overall, this could be something to investigate for remux. for now, we have to trust that the source code does what it says and that the community is not being malicious.

i'm not a security person, so i don't know how they grade security issues, but i think there are two major portions: opportunity size and risk size. the opportunity is low (not that many users: maybe dozens or low hundreds, i have no way of knowing) and the risk is also low (people do not keep sensitive personal information on their readers. the major risk is having your data deleted on you or someone install spyware to report on your activity). probably we mostly need to watch out for people who get some fun out of harming other people's daily life (a griefer)

[raisjn]

Originally Posted by elinkser

Hmm, so we at least had one testimony of rmkit working on the Elipsa 2E:
https://www.mobileread.com/forums/sh...46&postcount=3

And looking at the rest of the Kobo lineup, I see mostly Freescale (other than Sage/Elipsa):
https://en.m.wikipedia.org/wiki/Kobo_eReader

So can we infer that the other Kobos should support rmkit?


But there's something {among many other things) that still troubles me.
The Pocketbook lineup appears to be largely Allwinner, like the dreaded Sage/Elipsa:
https://en.m.wikipedia.org/wiki/Pock..._International

Yet by some feate of Magicke there has appeared fbink supporton the Pocketbook:
https://github.com/NiLuJe/FBInk

Some of the conjurations have even been revealed:
https://github.com/NiLuJe/FBInk/pull/64


So is there no way to use a similar incantation for rmkit??

There likely is, but I would prefer to use fbink as a backend for rmkit to draw to screen - just not sure how much work that will take. Typically, I talk to NiLuJe / look at fbink to learn how to support new devices. I believe that we have a reasonable coverage between the Clara, Libra and Elipsa 2E. For each new device, it's typically 1) figuring out if the fb needs special support, 2) figuring out any input event differences, 3) aligning the input coordinates to display coordinates

[raisjn]

Originally Posted by elinkser

POSSIBLE SECURITY ISSUES OF RMKIT


Since rmkit makes it easier to quickly develop cross-platform apps, some might roll their eyes and say, "Great, more junk apps".


Security-conscious people should also be concerned that a proliferation of low quality apps makes a nice target for malware.

-C++ requires effort and skill to be done correctly, and therefore securely.

-Kobo Nickel is designed for reading, not networking. It lacks up-to-date kernel, fine-grained permissions in the user partition, and kernel support for firewalls, among other issues.


How can this problem be addressed?
Kobo is even less equipped to deal with this than old Windows was.


Some random ill-thought out musings:

- Use InkBox instead of Nickel

- Only allow signed apps to launch in rmkit

- Run an app scanner over apps in the rmkit repository.

- Run a user-space firewall in rmkit (all i/o accesses only allowed through "secure" api)

- Require users who want to run Simple framework to self-sign any app they want to give run permission to.



Who would do all this work?
Not me - I'm not a dev, and definitely not a C++ dev.
But I want to enjoy my Kobo in a secure way.

These are real concerns, I appreciate it. I think e-readers generally run as root and are more exposed. I know Oxide (from rM) does put apps in a chroot, so that is a theoretical avenue. Other than that, I think there are two factors to consider: opportunity size (dozens to hundreds of people) and risk size (deleting your data, but sensitive information should mostly not be contained on device).

The source code is always exposed for rmkit apps, so you should take time to audit or trust that people are not trying to ruin your day. I really hope it continues to stay that way: the homebrew community here on mobileforums is awesome and there is a lot of trust (people just download and install somewhat opaque packages with binaries in them).

[elinkser]

@raisjn

I appreciate your consideration of my concerns.

I'm familiar enough about computer security to avoid using a cool network app to connect from my insecure kobo to my phone I use for banking, but I don't necessrily expect regular users to be.
They may have an expectation that they are being taken care of.


On a funner note, I finally got around to trying the remarkable port of Simon Tatham's Portable Puzzle Collection!
(I first tried the binary, then recompiled it for my Clara HD - attached).
mv puzzles.bin puzzles.xz
xz -d puzzles.xz


First tried the binary:

mrichards42/remarkable_puzzles
https://github.com/mrichards42/remarkable_puzzles

Simon Tatham's Portable Puzzle Collection for the reMarkable tablet.

$ mkdir mypuzzles

$ cd mypuzzles/

$ wget https://github.com/mrichards42/remar...puzzles.tar.gz

$ tar zxvf puzzles.tar.gz

On the kobo create etc/puzzles/ folder in /mnt/onboard/.adds/rmkit/.

Copy the config/, help/, icons/ folders to the /mnt/onboard/.adds/rmkit/etc/puzzles/ folder.

Copy the puzzles binary to the /mnt/onboard/.adds/rmkit/bin/apps/ folder.

/mnt/onboard/.adds/rmkit/bin/apps/puzzles: /lib/libm.so.6: version `GLIBC_2.27' not found (required by /mnt/onboard/.adds/rmkit/bin/apps/puzzles)


OK let's try to build:

$ git clone --recurse-submodules https://github.com/mrichards42/remarkable_puzzles.git

$ cd remarkable_puzzles/

$ source ~/koxtoolchain/refs/x-compile.sh kobo env bare

$ nano -l defs.mk

...
 11 export BUILD ?= release
...
 26 else ifeq ($(ARCH),kobo)
 27         CXX = arm-kobo-linux-gnueabihf-g++
 28         CC  = arm-kobo-linux-gnueabihf-gcc
 29         STB = stb.arm.o

$ nano -l vendor/rmkit/src/common.make

  3 CXX_BIN?=arm-kobo-linux-gnueabihf-g++
  4 CC_BIN?=arm-kobo-linux-gnueabihf-gcc
  5 STRIP_BIN?=arm-kobo-linux-gnueabihf-strip

$ nano -l vendor/rmkit/src/rmkit/input/events.cpy

241 //        case ABS_MT_SLOT:
242 //          slot = data.value;
243 //          break
...
282 //        case ABS_MT_DISTANCE:
283 //          slots[slot].distance = self.distance = data.value
284 //          if self.distance > 0:
285 //            self.left = 0
286 //          break

$ nano -l vendor/rmkit/src/rmkit/input/input.cpy

 69 //          if ev_data[i].code == SYN_DROPPED:
 70 //            syn_dropped = true
 71 //            event.handle_drop(fd)
 72 //            continue

$ nano -l vendor/rmkit/src/remux/launcher.cpy

 31 #define TOUCH_FLOOD_EVENT ABS_DISTANCE

$ ARCH=kobo make puzzles

$ cp build/release/puzzles .

$ arm-kobo-linux-gnueabihf-strip -s puzzles
$ ls -l puzzles
-rwxr-xr-x 1 1892712 puzzles

$ zip puzzles.zip puzzles
$ ls -l puzzles.zip
-rw-r--r-- 1 1119921 puzzles.zip

$ xz puzzles
$ ls -l puzzles.xz
-rwxr-xr-x 1 919152 puzzles.xz

On the kobo create etc/puzzles/ folder in /mnt/onboard/.adds/rmkit/.

Copy the config/, help/, icons/ folders to the /mnt/onboard/.adds/rmkit/etc/puzzles/ folder.

Copy the puzzles binary to the /mnt/onboard/.adds/rmkit/bin/apps/ folder.
mv puzzles.bin puzzles.xz
xz -d puzzles.xz

Works great, puzzle lovers rejoice!