(Project) Drawgress Bar — KUAL extension

[NiLuJe]

Yep, no arrays with the Kindle's busybox ash build.

Depending on your needs, you can fake arrays in a number of ways in POSIX shell (@frostchutz has done some fun stuff for Kobo like that, for instance, c.f., https://github.com/frostschutz/Kobo).

[mergen3107]

@NiLuJe
Thank you!

[mergen3107]

Here is some progress on the progressbar project :D
(follow ups are provided in bold - from 02/12/2020)

1) I created a GitHub repo. Current version archive is attached here as well.

2) Current version (1.87), unfortunately, is very raw.
It is supposed to be a KUAL extension, but for now I run the Drawgressbar.sh script from ssh, while Kindle has an open book.

3) How it works:
- the user manually inputs necessary book max length in locations, location steps during page turns, chapter and subchapter locations (obtained from ToC)
- the script calculates necessary dimensions, then draws the progressbar by eips.

It can only fetch the current location from /var/local/cc.db, because cc.db is updated only on book closing. It should, in future, be able to redraw the progressbar at every page turn by using a new location estimate. I think I should try to fetch a more precise new location that appears in 'showlog -f' at every page turn, but they seem to be in the units of html symbols or something instead of locations.

However here comes my main issue:
4) awk page turn detection code is not working (lines 231-232 in my script, commented out for now). I took this part from Showtime extension. Maybe this is related to bash binary, which I discussed above? I use it to be able to work with arrays in bash.
(the main issue was that I didn't export variables in bash part of the script. Once I did, all the variables are available as ENVRION["variable"] in awk itself, and system("echo $variable") in the system access through awk. )

Also, I don't know how to feed a multiline argument for that awk action.
Each desired line break column should be "\", and NO other new white spaces should be added. e.g.

awk '/.*pattern1*./ {system("echo \
$variable")} '

5) I don't know how to catch 'page turn forward' and 'page turn backward' actions separately.

&> /var/log/messages && 2>/dev/null tail -F /var/log/messages | exec parselog | \
awk '/.*PageTurn=Next*./ {action1} /.*PageTurn=Previous*./ {action2} '

6) The script has patches of python and different workarounds (bash is terrible in working with decimals!) here and there, so I will probably switch to python completely later. At least for the calculation part, because I don't know how to access eips from python.

7) Can you please explain me about the need in any open source licenses? I know it is just an extension for KUAL, but since this is my first large public project, I would like to properly acknowledge all parts of work I used and make sure everyone can benefit from this project. I wrote a made-up terms in the beginning of the script for now.

Guys can you please help me to finish this project until at least a properly working extension? :)

Thank you!

[jhowell]

Originally Posted by mergen3107

It can only fetch the current location from /var/local/cc.db, because cc.db is updated only on book closing. It should, in future, be able to redraw the progressbar at every page turn by using a new location estimate. I think I should try to fetch a more precise new location that appears in 'showlog -f' at every page turn, but they seem to be in the units of html symbols or something instead of locations.

See this thread for information on parsing files that contain reading progress and are (AFAIK) updated with every page turn: KRDS - A parser for Kindle reader data store files

(Added: Post #7 in that thread has a little info about how to interpret reading positions.)

(Further added: The translation table between "positions" and "locations" in KF8 (azw3) format is contained in the DATP MOBI record.)

[NiLuJe]

@mergen3107:

In no particular order .

4 & 5/ While it's probably a viable approach (I'd double-check if the log has changed formatting), I'd "simply" (he said ;p) listen for input events, or "poll" a sentinel pixel (àla MiniClock).

6/ That kind of slipped my mind before, but USBNet ships with a full ZSH build, which can do all of those things properly, and more, because ZSH is awesome.

But, going with Python will probably save you some pain, so: https://github.com/NiLuJe/py-fbink.
You'll also have proper SQLite bindings with Python, too.

(A somewhat straightforward way to draw filled rectangles (via cls + background color) has been made public in the current FBInk tree, which'll hit a release Soon(TM)).

7/ Spoiler warning: IANAL, and this is an extremely abridged Cliff's Notes version.

On two opposite edges:

GPL: This is FOSS, I wrote it, and I care about how it's being (re-)used.
MIT: This is FOSS, I wrote it, and you can do whatever the fuck you want with it.

And a myriad of different licenses in-between .

Regardless of that, I find human-readable (i.e, not lawyer speak) acknowledgements to be the right thing to do. I usually go with a CREDITS file, or tuck that somewhere in the README.

[mergen3107]

Thank you so much @jhowell @NiLuJe!
I’ll see what I can do

[mergen3107]

OK, some news.

1) I found my blatant mistake with awk: I didn't export variables used later in the awk. I found it just by manually trying to run the code line by line... Still working to finish the first draft of a functional code.

2) Now, I have no clue how to run different awk 'triggers' simultaneously. I need separate triggers for 'page forward', and 'page backward', at least. The brute force way in my mind is to run a separate script for each.

3) @NiLuJe, can you please support/crash my following idea:
— I found one of your projects to literally "draw" clock on the screen.
— I am thinking that the frame of the progress bar (chapter and subchapter marks + the beginning and the end marks, maybe) is not changing from page to page. The only thing is changing, actually, is the current progress line.
— So, what if I draw the frame once at the script execution (when (sub)chapter locations are processed), then fbrab it and save as a png, and just draw it by fbink each time needed?
— The only potential obstacle now is whether transparency is supported in fbgrab/convert (or Kindle at all).
— At every page turn, I'd draw a) current progress fillers first, then b) on top of it, the frame with (sub)chapters marks with 'alpha channel gaps' in between. This way I'd save multiple loops for drawing the same (sub)chapter marks (and I don't know how to implement loops in the awk actions anyway )

Thank you in advance!
I appreciate all your support

P. S. If no transparency is possible, I will attempt to make a rough progress bar, where the frame is drawn in a separate box from the progress bar...

[NiLuJe]

That was a terrible, terrible, terrible PoC .

Actual mostly-sane implementation of the same idea, on, Kobo: https://github.com/NiLuJe/Kobo/blob/...k/miniclock.sh

----

That said, for your use-case, that's vaguely less terrible, at first glance.

It'd be much easier/cleaner/efficient without the PNG round-trip (i.e., via fbink_dump/fbink_restore), but that's API only, can't do that in shell .

In fact, you have a major issue with the fbgrab approach: it'll grab the full screen. You don't want the full screen, you only want the bar.
And you can't print only a subsection of an image.
So you'd have to crop your image manually and separately and save *that* in another PNG.
That probably means ImageMagick has to come in, and the idea kind of has become a giant soul-sucking monster ^^.

I'm not quite sure I get where transparency would come in, though?

But, FWIW, there won't be any alpha channel on your screengrab, because it'll be grayscale. FBInk, on the other hand, would honor alpha & blend properly if there was.

----

With that out of the way, I think you might be overthinking this:

* Drawing rectangles is *super* cheap.
* You can draw without refreshing the screen (-b, --norefresh).

Which means:

Just imagine your bar components as a stack, and draw it from back to front:

* Frame
* Background
* Filler
* Markers

(This is far more efficient, and easier, than drawing it "left to right" in mixed chunks. Think layers of a cake, except it's a cake you only get to see from straight overhead ^^).

Then trigger a refresh for the region that encompasses the full bar, and voilà.
(Pro tip: you can probably use the region from a get_last_rect from the initial frame drawing for that).

That's how FBInk handles its own progress/activity bar drawing (e.g, activity bar).

(Now, if *computing* your rectangles is non-trivial, and some of those are fixed, just cache the computed values ).

[NiLuJe]

Random example of the previous message in ZSH (yeah, I was bored. On the upside, I found a bug in FBInk, so, eeeh ).

#!/usr/bin/env zsh

# Debug
set -x

# For rint
zmodload zsh/mathfunc

# For screen dimensions & stuff
eval $(fbink -qe)

# Piggyback on default font scaling for a bar height
barHeight=${FONTH}

# Compute bar width as 80% of screen's width
integer barWidth=$(( rint(screenWidth * 0.8) ))

# Tiny bottom margin (1% of screen's height)
integer bottomMargin=$(( rint(screenHeight * 0.001) ))

# Compute the bar's top-left corner coordinates
integer barTop=$(( rint(screenHeight - bottomMargin - barHeight) ))
integer barLeft=$(( rint((screenWidth - barWidth) / 2.) ))

# Colors
frameCol="WHITE"
bgCol="GRAY9"
fillCol="BLACK"

# Draw the frame (and remember the draw area)
# NOTE: Oops, -E + -k was broken until 01aadd1
#       On Kindle, we can just re-use bar* instead anyway ;).
#eval $(fbink -E -b -k top=${barTop},left=${barLeft},width=${barWidth},height=${barHeight} -B ${frameCol})
fbink -q -b -k top=${barTop},left=${barLeft},width=${barWidth},height=${barHeight} -B ${frameCol}

# Draw background bar (given a 1px frame on each side)
integer fillTop=$(( barTop + 1 ))
integer fillLeft=$(( barLeft + 1 ))
integer fillWidth=$(( barWidth - 2 ))
integer fillHeight=$(( barHeight - 2 ))

fbink -q -b -k top=${fillTop},left=${fillLeft},width=${fillWidth},height=${fillHeight} -B ${bgCol}


# Draw progress bar (current reading position at 42%)
readPos=0.42
integer readWidth=$(( rint(fillWidth * readPos) ))

fbink -q -b -k top=${fillTop},left=${fillLeft},width=${readWidth},height=${fillHeight} -B ${fillCol}


# Compute & draw chapter markers
integer tickWidth=$(( rint(screenWidth * 0.0015) ))
chapters=(0.20 0.45 0.72)
for chapter in ${chapters[@]} ; do
	integer tickLeft=$(( rint(fillLeft + (fillWidth * chapter)) ))

	fbink -q -b -k top=${barTop},left=${tickLeft},width=${tickWidth},height=${barHeight} -B ${frameCol}
done


# Compute & draw subchapter markers (square, middle of the bar)
integer tickPx=$(( rint(screenWidth * 0.002) ))
integer tickTop=$(( rint(fillTop + (fillHeight / 2.) - (tickPx / 2.)) ))
chapters=(0.05 0.11 0.30 0.86 0.92)
for chapter in ${chapters[@]} ; do
	integer tickLeft=$(( rint(fillLeft + (fillWidth * chapter)) ))

	fbink -q -b -k top=${tickTop},left=${tickLeft},width=${tickPx},height=${tickPx} -B ${frameCol}
done


# Refresh the bar
#fbink -q -s top=${lastRect_Top},left=${lastRect_Left},width=${lastRect_Width},height=${lastRect_Height}
fbink -q -s top=${barTop},left=${barLeft},width=${barWidth},height=${barHeight}

Should be easy to adapt in Python/Lua/Go, there's nothing fancier than arrays, fp maths and int()/rint() (hence ZSH instead of shell).

[mergen3107]

Thank you for a good example!
I finished a very basic example of drawing current progress on page turns. It now recognizes forward/backward page turns and corrects the current location properly. I will upload it here soon (see new post down below).
Then I’ll try to optimize all calculations (and it looks like I can do everything in awk, the only thing I can’t do right now is rounding floats to integers, so it is just taking integer parts via int() for now. I found an example function round.awk on the official AWK user guide website, but couldn’t include it into the awk line just yet).

[mergen3107]

Here is v1.99A (warning: this is beta-beta-beta version)
At least it can draw current location estimate at every page turn:

0) Copy it to extensions as usual
1) Open a desired book
2) Replace chapter/subchapter locations, book length in the DrawgressBar.sh
3) Run it from KUAL (preferably via "Start DrawgressBar" because it also logs outputs
4) Start reading the book. The progress bar should appear at each page turn.
5) after usage hit “Stop Drawgressbar” button in KUAL to stop log reading
It only shows finished/unfinished parts right now. Will add chapters later.

Any suggestions are welcome!

@NiLuJe, I wasn't able to draw via "fbink -b -q -k" or "fbink -bqk", however the fbink output shows usual info as if it is drawing. "fbink -qk", on the other hand, works, but I don't notice screen refresh anyway.
Also, it looks to me that fbink is drawing as if it "tearing". Like, it draws dark widths larger first, then adjusts.

P.S. b2t.sh and b2t_2.sh script contain just my tests of various functions, mostly awk. awk is beautiful and neat!

[NiLuJe]

@mergen3107: -b just draws with no refresh, that's the whole point (hence the final -s call in the example above) . Because as you've noticed, if you start mixing colors other than B&W in a bar with incremental refreshes in quick succession, things start to look wonky.

Note that the EPDC itself will probably inhibit refreshes if the content hasn't been changed, given the waveform mode being used (because AUTO will probably default to DU given the type of content being drawn).

(e.g., if you repeat fbink -Mm 'Wheee' -W DU, the subsequent calls won't actually refresh the screen, because there's nothing new to show).

[mergen3107]

So I should "draw" everything (chapters/subchapter, finished/unfinished background) with -b first, and then refresh via -s the whole progress bar rectangle?

[mergen3107]

@NiLuJe,
I tried this sequence: 1) fbink -q -b -k dimensions color ("draw" things); 2) fbink -q -s full_dimensions (refresh screen). However I still see 'tearing' drawing in my script, as well as in yours. Do I have a wrong expectation, waiting for just one "refresh" that would reveal the fully drawn progress bar?
I still see the latest release on github being from 10 days ago.

By the way, the fbink version is FBInk v1.21.0-1-g747530d for Kindle (the first one after you announced v1.21.0 I guess). Is it too old to be affected by your message "# NOTE: Oops, -E + -k was broken until 01aadd1"?

[NiLuJe]

@mergen3107: Then that's (probably?) just eInk being eInk . Older screens can be quirky with specific shades of grays in direct contrast with pure black/white.

I haven't noticed anything out of the ordinary on a H2O or a Forma. But, I can see what you mean on a PW2 (i.e., that the whole width of the bar flashes black first, despite not requesting a flashing update), and I don't think there's anything you can do about it (short of switching to pure B&W and A2 or DU).

(i.e., even -W DU4 -D LEGACY is affected. Only -W DU -D LEGACY will work, because it's B&W. But the Y2 dithering on Kindle is, err, not great, so you probably don't want to use that ;D).

----

Yeah, the 1.21.0 release is affected by the issue mentioned, since I fixed it only last night .
That specific usecase turned out to be too roundabout for its own good on Kindle anyway, as the comment explains .