Per app eink "optimizations"

[Trender22]

Originally Posted by Renate

Oh, I hadn't noticed that we were talking about an mmkv file. My Go6 only has /onyxconfig/eac_config which is a (bloated) JSON file.

in 4.0 firmware they switched to using mmkv evidently - see comment from starkruzr.
also last comment here as well: https://gist.github.com/calliecamero...8bd493380e3b7e

[Renate]

Yeah. There are also enough mmkv files in 3.5 to play with, just not eac_config.

BOOX:/data/data # find . -name '*mmkv' | xargs ls -l
-rwx------ 1 system system 8192 2024-10-28 05:59 cloud_note
-rwx------ 1 system system 4096 2024-10-28 05:59 cloud_note.crc
...

Lol. I don't know how your mmkv eac_config is but the mmkv's I have are pretty stupid. The idea of mmkv is supposed to be lean and mean key/value. If all your values are JSON structures formatted with whitespace there's not a lot of savings.

[starkruzr]

Originally Posted by Trender22

Thanks! will try that shortly.
Meanwhile here is the list of processes that actually work with that config file (obtained by running lsof in adb shell after su):

COMMAND     PID       USER   FD      TYPE             DEVICE  SIZE/OFF       NODE NAME
system_server  1548     system  mem       REG                8,9    524288         13 /onyxconfig/mmkv/onyx_config
system_server  1548     system  200u      REG                8,9    524288         13 /onyxconfig/mmkv/onyx_config
ndroid.systemui  1853     u0_a89  mem       REG                8,9    524288         13 /onyxconfig/mmkv/onyx_config
ndroid.systemui  1853     u0_a89  127u      REG                8,9    524288         13 /onyxconfig/mmkv/onyx_config
com.onyx   3139     system  mem       REG                8,9    524288         13 /onyxconfig/mmkv/onyx_config
com.onyx   3139     system  319u      REG                8,9    524288         13 /onyxconfig/mmkv/onyx_config

my sense is that the configs being written into /onyxconfig/mmkv/onyx_config must be coming from some internal database somewhere. I think it might be that thing that we actually want to edit.

[Renate]

No, the mmkv files are the actual "database". I made a "walker":

Key: GOOGLE_DRIVE_CLIENT_KEY   // There are 15 copies of this which are inactive
Value 406 bytes:
{
  "installed": {
    "client_id": "long-secret-string.apps.googleusercontent.com",
    "project_id": "kreader-1234567890",
    "auth_uri": "https://accounts.google.com/o/oauth2/auth",
    "token_uri": "https://oauth2.googleapis.com/token",
    "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
    "client_secret": "24-chars-gobble-dee-gook"
  }
}
Key: GOOGLE_DRIVE_CLIENT_KEY   // This is the only active data in the whole file
Value 406 bytes:
{
  "installed": {
    "client_id": "long-secret-string.apps.googleusercontent.com",
    "project_id": "kreader-1234567890",
    "auth_uri": "https://accounts.google.com/o/oauth2/auth",
    "token_uri": "https://oauth2.googleapis.com/token",
    "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
    "client_secret": "24-chars-gobble-dee-gook"
  }
}Key: GOOGLE_DRIVE_CLIENT_KEY   // There are 2 copies of this which are inactive
Value 406 bytes:
{
  "installed": {
    "client_id": "long-secret-string.apps.googleusercontent.com",
    "project_id": "kreader-1234567890",
    "auth_uri": "https://accounts.google.com/o/oauth2/auth",
    "token_uri": "https://oauth2.googleapis.com/token",
    "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
    "client_secret": "24-chars-gobble-dee-gook"
  }
}

So you have 18 copies of the same thing, one copy marked as active.

[Trender22]

Originally Posted by Renate

No, the mmkv files are the actual "database".
So you have 18 copies of the same thing, one copy marked as active.

It's pretty common pattern in non-SQL DBs these days to just keep adding snapshots of values as it results in better write performance. And reading in this case is probably only done once during initialization and then whole set of active values is kept in memory.
While experimenting with onyx_config file I ended up with setup when my custom file was there, but was owned by 'root' rather than 'system' and was not witeable. After reboot in this setup dmesg showed error while starting 'init_onyx_config' service.

Since then I rectified my mistake and now all handwriting optimizations listed here (https://gist.github.com/calliecamero...ent_id=4993606) are enabled.

I had to download onyx_config along with crc file to a laptop and use mmkv-cli to extract configs of apps in question, edit them according to the article linked above and then set back into the DB file using mmkv-cli again. Note: when setting I had to 'escape' all double quotes of the JSON with backslash so that it worked properly in command line.

Then I transferred updated file back to the device and replaced original with it.
To avoid system applications overriding my version of the file I renamed original (mv command) - they kept their file handles pointing to that renamed original until reboot.
Once I placed my version in /onyxconfig/mmkv I also had to chmod it to 777 and chown/chgrp it to system. After reboot all apps that I touched in the config had the Handwriting tab in the Optimization popup.

I suppose it'd be nicer to port mmkv-cli to android shell or write an app using mmkv lib natively, but I've never developed or even just compiled anything for android so was happy to use mmkv-cli as suggested above.

[starkruzr]

Originally Posted by Trender22

I had to download onyx_config along with crc file to a laptop and use mmkv-cli to extract configs of apps in question, edit them according to the article linked above and then set back into the DB file using mmkv-cli again. Note: when setting I had to 'escape' all double quotes of the JSON with backslash so that it worked properly in command line.

wait, you got this working?? I did exactly the same thing and mmkv-cli couldn't recognize anything.

[Trender22]

Originally Posted by starkruzr

wait, you got this working?? I did exactly the same thing and mmkv-cli couldn't recognize anything.

yep, works fine like this:

mmkv-cli get eac_app_com.dragonnest.drawnote --id onyx_config

(executed in the directory where you put both onyx_config and onyx_config.crc)

the "--id" option is important.

--id [string]
MMKV instance id. If not specified, default is used.

[starkruzr]

this is fantastic. are there other DrawViewKeys you've discovered and used?

[Trender22]

Originally Posted by starkruzr

this is fantastic. are there other DrawViewKeys you've discovered and used?

I have not discovered anything new, just configured the ones listed here: https://gist.github.com/calliecamero...ent_id=4993606

It works well for handwriting in Xodo, but in general this is still very much a hack - don't expect perfect user experience.
if handwirting optimization is enabled any operation in the app with a pen results in drawing on a screen with default color. Even if I try to press a button with a pen somewhere in Obsidian UI it results with a dot drawn on top of that button and no action - I have to fall back to using finger.
If I'm in some drawing app with this optimization enabled and I'm using some shapes rather than pen/pencil/brush - again there will be first a line drawn and then I have to touch screen and it transforms into the shape. So if I'm adding a rectangle Onyx hack will first draw its diagonal following motion of my pen and then the app will transform it to a rectangle.
OneNote seems to have better integration:
1. it informs Onyx hack of current color - so the scribble created by Onyx hack is at least of the color selected by OneNote
2. it seems to inform Onyx hack that drawing mode is over and then pen behaves normally until I enter drawing mode in OneNote again.

So I think to make better use of this we need:
1. a shortcut for enabling / disabling handwriting optimization in one click
2. explore additional settings within the handwriting optimization config, see what values are specified for OneNote - maybe some of the fancier OneNote features can be enabled for other apps

For my handwriting use cases however I have what I need in Xodo. If I get time I'll see if I can enable this hack for GBoard, because it is so much better at recognizing my scribbles than Onyx keyboard.

[Trender22]

To further elaborate on one of the points above here is what config for OneNote looks like. Clearly styleMap allows to recognise names of styles in OneNote terms into specific config for the Onyx hack. However to figure out what those style names would be for other apps (even if they do communicate those styles at all) would require debugging if not decompiling the code:

    "noteConfig" : {
      "compatibleVersionCode" : 0,
      "enable" : true,
      "styleMap" : {
        "highlighter_1" : [
          {
            "type" : 2,
            "value" : "-248"
          },
          {
            "type" : 0,
            "value" : "2"
          },
          {
            "type" : 1,
            "value" : "60.0"
          }
        ],
        "pen_3" : [
          {
            "type" : 2,
            "value" : "-12865536"
          },
          {
            "type" : 1,
            "value" : "10"
          },
          {
            "type" : 0,
            "value" : "1"
          }
        ],
        "highlighter_2" : [
          {
            "type" : 2,
            "value" : "-14417921"
          },
          {
            "type" : 1,
            "value" : "60.0"
          },
          {
            "type" : 0,
            "value" : "2"
          }
        ],
        "pen_2" : [
          {
            "type" : 0,
            "value" : "1"
          },
          {
            "type" : 2,
            "value" : "-2883566"
          },
          {
            "type" : 1,
            "value" : "6.0"
          }
        ],
        "pen_1" : [
          {
            "type" : 1,
            "value" : "6.0"
          },
          {
            "type" : 2,
            "value" : "-156916"
          },
          {
            "type" : 0,
            "value" : "1"
          }
        ],
        "pen_4" : [
          {
            "type" : 1,
            "value" : "12.0"
          },
          {
            "type" : 2,
            "value" : "-16572072"
          },
          {
            "type" : 0,
            "value" : "1"
          }
        ]
      },
      "supportNoteConfig" : true,
      "repaintLatency" : 500,
      "globalStrokeStyle" : {
        "strokeWidth" : 6,
        "enable" : true,
        "strokeColor" : -16777216,
        "strokeExtraArgs" : [

        ],
        "strokeStyle" : 1,
        "strokeParams" : [

        ]
      },
      "drawViewKey" : "com.microsoft.office.airspace.AirspaceInkLayer"
    },

[starkruzr]

Originally Posted by Trender22

yep, works fine like this:

mmkv-cli get eac_app_com.dragonnest.drawnote --id onyx_config

(executed in the directory where you put both onyx_config and onyx_config.crc)

the "--id" option is important.

--id [string]
MMKV instance id. If not specified, default is used.

so I tried your example and it worked:

❯ mmkv-cli get eac_app_com.dragonnest.drawnote --id onyx_config
{"activityConfigMap":{},"autoFreezeConfig":{"autoFreeze":false,"enable":true,"supportAutoFreeze":true},"autoStartConfig":{"autoStart":false,"enable":true,"supportAdjust":true},"colorConfigMap":{},"cssConfigMap":{},"dpiConfig":{"dpi":450,"enable":true},"enable":true,"extraConfig":{"allowSplashScreen":false,"enable":true,"forceFullScreen":false,"fullPMAccess":false,"useDialogBorder":false,"usePageKeyAsVolumeKey":true},"forceScrollRefreshClsList":[],"globalActivityConfig":{"clsName":"","disableScrollAnim":false,"displayConfig":{"bwMode":0,"cfaColorBrightness":0,"cfaColorSaturation":0,"cfaColorSaturationMin":0,"contrast":30,"ditherThreshold":255,"enable":true,"enhance":true,"monoLevel":10},"enable":true,"noteConfig":{"compatibleVersionCode":0,"enable":false,"globalStrokeStyle":{"enable":false,"strokeColor":-16777216,"strokeExtraArgs":[],"strokeParams":[],"strokeStyle":0,"strokeWidth":3},"repaintLatency":500,"styleMap":{},"supportNoteConfig":false},"paintConfig":{"antiAlisingType":0,"ditherBitmap":false,"enable":true,"fillBrightness":0,"fillContrast":0,"fillEAC":false,"iconBrightness":0,"iconContrast":0,"iconEAC":false,"iconThreshold":0,"imgEAC":true,"imgGamma":60,"quantBits":3,"textBold":false,"textEACType":0},"refreshConfig":{"animationDuration":50,"antiFlicker":10,"enable":true,"gcInterval":20,"supportRegal":false,"turbo":0,"updateMode":2,"useGCForNewSurface":false}},"globalCSSConfig":{"clsName":"","customCSS":"","enable":true,"fontBold":false,"fontColor":0,"fontSize":100},"keyboardConfig":{"enable":true,"pageKeyMode":1},"networkConfig":{"enable":true,"overrideRules":0},"pkgName":"com.dragonnest.drawnote","rotationConfig":{"enable":false,"overrideRotation":-1},"scrollArgs":{"duration":600,"endXPercent":0.5,"endYPercent":0.8,"sampleTime":16,"startXPercent":0.5,"startYPercent":0.2},"supportEAC":true}

and I can get it to pretty-print it by piping it through jq, which is nice.

what I'm not clear on is how to use "set" in mmkv-cli to change these settings. do you have any examples of this?

[Trender22]

set works naturally. when exporting I did not do pretty print and edited configuration json as it was - in a single line. Then I escaped the quotation marks with backslashes and used 'set' command to put it back into the store

I had to download onyx_config along with crc file to a laptop and use mmkv-cli to extract configs of apps in question, edit them according to the article linked above and then set back into the DB file using mmkv-cli again. Note: when setting I had to 'escape' all double quotes of the JSON with backslash so that it worked properly in command line.

[Trender22]

It works like this - whole json in single line with double quotes escaped by backslashes:

mmkv-cli set eac_app_md.obsidian "{\"activityConfigMap\":{},\"autoFreezeConfig\":{\"autoFreeze\":false,\"enable\":true,\"supportAutoFreeze\":true},\"autoStartConfig\":{\"autoStart\":false,\"enable\":true,\"supportAdjust\":true},\"colorConfigMap\":{},\"colorMode\":\"vivid\",\"cssConfigMap\":{},\"dpiConfig\":{\"dpi\":450,\"enable\":true},\"enable\":true,\"extraConfig\":{\"allowSplashScreen\":false,\"enable\":true,\"forceFullScreen\":false,\"fullPMAccess\":false,\"useDialogBorder\":false,\"usePageKeyAsVolumeKey\":true},\"forceScrollRefreshClsList\":[],\"globalActivityConfig\":{\"clsName\":\"\",\"disableScrollAnim\":false,\"displayConfig\":{\"bwMode\":0,\"cfaColorBrightness\":0,\"cfaColorSaturation\":50,\"cfaColorSaturationMin\":60,\"contrast\":30,\"ditherThreshold\":128,\"enable\":true,\"enhance\":true,\"monoLevel\":10},\"enable\":true,\"noteConfig\":{\"compatibleVersionCode\":0,\"drawViewKey\":\"com.getcapacitor.CapacitorWebView\",\"enable\":true,\"globalStrokeStyle\":{\"enable\":true,\"strokeColor\":-16777216,\"strokeExtraArgs\":[],\"strokeParams\":[],\"strokeStyle\":0,\"strokeWidth\":3},\"repaintLatency\":500,\"styleMap\":{},\"supportNoteConfig\":true},\"paintConfig\":{\"antiAlisingType\":0,\"ditherBitmap\":false,\"enable\":true,\"fillBrightness\":0,\"fillContrast\":0,\"fillEAC\":false,\"iconBrightness\":0,\"iconContrast\":0,\"iconEAC\":false,\"iconThreshold\":0,\"imgEAC\":true,\"imgGamma\":60,\"quantBits\":3,\"textBold\":false,\"textEACType\":0},\"refreshConfig\":{\"animationDuration\":50,\"antiFlicker\":10,\"enable\":true,\"gcInterval\":20,\"supportRegal\":false,\"turbo\":2,\"updateMode\":2,\"useGCForNewSurface\":false}},\"globalCSSConfig\":{\"clsName\":\"\",\"customCSS\":\"\",\"enable\":true,\"fontBold\":false,\"fontColor\":0,\"fontSize\":100},\"keyboardConfig\":{\"enable\":true,\"pageKeyMode\":1},\"networkConfig\":{\"enable\":true,\"overrideRules\":0},\"pkgName\":\"md.obsidian\",\"rotationConfig\":{\"enable\":false,\"overrideRotation\":-1},\"scrollArgs\":{\"duration\":600,\"endXPercent\":0.5,\"endYPercent\":0.8,\"sampleTime\":16,\"startXPercent\":0.5,\"startYPercent\":0.2},\"supportEAC\":true}" --id onyx_config

[Trender22]

Originally Posted by starkruzr

and I can get it to pretty-print it by piping it through jq, which is nice.

mmkv-cli has built-in formatter option:

--prettify-json | -p

[calliecameron]

It looks like mmkv-cli only supports macOS - on linux I was able to make the changes using mmkv's python bindings. Unfortunately it's a lot less convenient.