Detecting if book being read / home screen in nickel

[sv2000]

Hi,

I am trying to build an alternative way to launch koreader/other programs from within nickel, without relying on file monitoring.

My idea was to poll the /dev/input/event1 to check a particular region of the screen, and performing the desired action if touch coordinates are detected. Possibly print a small icon in the statusbar (if its possible to print to screen), which could be touched.

However I need a way to detect if the statusbar is hidden / we are reading something or if we are on the homescreen. Is there a way to accomplish something like this?

[NiLuJe]

I'm obviously fairly biased here, but this sounds fairly more complicated than inotify monitoring for no real good reason?

Unless the reason is fun, in which case: break a leg, the more the merrier .


In no particular order:

* There are three or four variants of touch protocols used on Kobos, and the original Touch trilogy is a fantastic mess to get right.

* Screen printing is easy enough (Hi!).

* As is screen scraping (Hi again!), but you kind of have to find something to match against. We went with solid strips of colors, because that's easy enough to do, and hopefully universal enough.

@frostschutz went with a full scanline here, which is trickier, especially if font rendering is involved (because your pattern will basically be device AND FW-specific, or nearly so).

* Actually playing nice with Nickel may be tricky, depending on how you end up running things and exactly how much control & safety you want to enforce (c.f., KFMon's teething pains in its infancy on that front).

----

I mentioned screen scraping, because getting anything interesting out of Nickel is pretty hard, so, that's one obvious solution.

The other is "simply" monkey patching Nickel at run-time to catch the relevant function calls, but in that case you can probably forget about anything else and just go with that .
(c.f., a few random posts on that subject over here).

[NiLuJe]

You might also be able to suss something out of the database's activity log, although I'm not sure if the Home screen actually triggers anything interesting there...

[sv2000]

Hi NiLuJe,

Thanks for the reply. I must admit this might not be the most well thought out idea. The fmon launcher icon gets buried beneath other books I read in nickel and I don't think there's a way to pin it to the home screen. Minor inconvenience, but I'm just starting to tinker with my Kobo Glo HD, and this was the first thing which came to my mind.

Thanks for the link to your screen painting library.

Screen scraping sounds like a good idea. I could scrape the screen to check for the presence of the battery icon (indicating menu bar is visible), and draw an icon (and run the touch detection routine). Not sure how this will affect battery life though.

As far as compatibility goes, while I would love to share anything I come up with, with the community, I'm doing this for just some random tinkering in my free time, and would be happy if I can just manage to make it work on my Glo HD.

If nickel can be patched obviously thats the better and direct solution, something like replace the store link with the launcher. However patching nickel with custom functionality is above my pay grade (I'm primarily a HW/EE person with limited coding skills ). Scraping the screen definitely sounds like something with my reach, wonder if you have any suggestions to monitor the screen in a power efficient way?

[davidfor]

Originally Posted by sv2000

Thanks for the reply. I must admit this might not be the most well thought out idea. The fmon launcher icon gets buried beneath other books I read in nickel and I don't think there's a way to pin it to the home screen. Minor inconvenience, but I'm just starting to tinker with my Kobo Glo HD, and this was the first thing which came to my mind.

The books on the home screen are all books being read with in order of most recently read. You could put a book to the first location on the home screen by setting the last read date to sometime in the future and setting the appropriate status. But, as soon as you opened the book, these would change. And I suspect that the fmon icon gets marked as read when it is opened. But, you could use a database trigger to do it.

Something like the following should do it:

CREATE TRIGGER PinFMonOnHome
AFTER UPDATE ON content
  FOR EACH ROW
  WHEN ContentID = 'file:///mnt/onboard/path_to_the_icon'
  AND ContentType = 6
BEGIN
  UPDATE content 
  SET DateLastRead = '2222-01-01T00:00:01Z, ReadStatus = 1
  WHERE rowid = new.rowid;
END

I haven't tested that. I'm working partly from memory and partly from looking at other examples I have posted here before and are in my Kobo Utilities plugin. I'm not going to guarantee that I have the syntax correct. But, it should be pretty close.

There is also a timing issue. While the books on the home screen are based on the ReadStatus and DateLastRead, it is possible that the database isn't always used. The device maintains the data in memory, so that it doesn't have to constantly reread the database. A trigger like this might only work when the device is turned on. Or it might work when you return to the home screen from the lists. It won't surprise me too much if when you finished a book, it didn't work but the next time the home screen was refreshed, that the book reappeared.

If you want to try this, take a copy of the database first. I might try it when I get home using one of my other devices. I'm not going to break my Clara HD just before my commute home

[NiLuJe]

@davidfor: Yeah, the timing issue is also compounded by the fact that if you're launching something that kills Nickel, that might effectively rollback (well, discard with WAL) the last few events related to the icon you've just clicked anyway .
(I readily admit to never having checked how well nickel copes with SIGTERM in terms of DB handling, so this point may well be moot ).

But that's definitely worth looking into nonetheless!

EDIT: And, yeah, the icon gets marked as Finished (or 99% Read for stuff that kills Nickel) .

@sv2000: My attempts to poll the screen refreshes were met with utter failure, so I guess I'd take the reverse approach: poll the touch input, and only scrape when you detect a touch in your area of interest.

EDIT: Wait, chicken & egg issue if you want to draw the icon *first*, as an indicator of where to touch... Ooops. .

EDIT²: And not much luck either with the DB, in terms of figuring out where we are (especially as far as the Home screen is concerned), either via SELECT * FROM Activity ORDER BY Date; or SELECT * FROM Event ORDER BY LastOccurrence; ;/.

EDIT³: Some more luck w/ SELECT * FROM AnalyticsEvents ORDER BY Timestamp;, except that it seems to be updated in short bursts, and not instantaneously :?

[pazos]

Originally Posted by NiLuJe

EDIT: And, yeah, the icon gets marked as Finished (or 99% Read for stuff that kills Nickel) .

This is for the first time the action is triggered. If you mark the book as unread and launch the png again you'll see an 1% reading

@sv2000: if you replace fmon with kfmon you can update nickel DB with some custom title/author and sort your library accordly.

[NiLuJe]

FWIW: my usual workflow is simply the same than with any other book on the Kobo: use the Library Search function, it'll honor the PNG filename as title even without KFMon's DB tinkering .

I guess if I actually used Shelves, I could also put all my ducks in a dedicated shelf via Calibre/Kobo Utilities . That's actually what I used to do on Kindle with Kindlets.
(My lack of interest in the shelf feature is *also* thanks to the Kindle getting increasingly annoying to deal with in terms of collection management, and me just going "fuck it, I barely use those anyway" and just living without collections ).

[davidfor]

Originally Posted by NiLuJe

@davidfor: Yeah, the timing issue is also compounded by the fact that if you're launching something that kills Nickel, that might effectively rollback (well, discard with WAL) the last few events related to the icon you've just clicked anyway .
(I readily admit to never having checked how well nickel copes with SIGTERM in terms of DB handling, so this point may well be moot ).

But that's definitely worth looking into nonetheless!

EDIT: And, yeah, the icon gets marked as Finished (or 99% Read for stuff that kills Nickel) .

No, that's expected. Nickel tends to mark non-kepubs as finished, but leaves the percent read as 99. I think that when it marks an epub as read, it doesn't update the percent read and just leaves it at whatever was last set.

EDIT²: And not much luck either with the DB, in terms of figuring out where we are (especially as far as the Home screen is concerned), either via SELECT * FROM Activity ORDER BY Date; or SELECT * FROM Event ORDER BY LastOccurrence; ;/.

The Activity table wouldn't be useful enough. For the Event, you would have to look for what was the last event out of things like page turn (46), open a book (1020), close a book (1021) and probably a couple of others.

EDIT³: Some more luck w/ SELECT * FROM AnalyticsEvents ORDER BY Timestamp;, except that it seems to be updated in short bursts, and not instantaneously :?

I haven't paid much attention to the AnalyticsEvents table. But, it gets cleared periodically, so it might not be helpful.

[sherman]

I've been running strace to see if I could spot some file opening patterns here. Here's what I've found for now. (note, running FW 4.8.11073)

Opening and closing an epub is pretty glaringly obvious, as Qt spawns some ipc files in /tmp/. More specifically, "/tmp/qipc_sharedmemory_comkoboadobepixmapsurface<hash?> and "/tmp/qipc_systemsem_comkoboadobepixmapsurface<hash?> only exist when an epub is open. Note, <hash?> is a long string of hex that doesn't change between reboots, but could differ between devices/firmwares).

Opening a kepub seems to trigger Nickel to search for mime/packages in a few directories such as /usr/<local>/share.

Opening the home screen (not the book-list) seems to trigger Nickel to look for /mnt/onboard/.kobo/guide and /mnt/sd/koboExtStorage/guide. In my poking around, I haven't got it to try accessing the guide dir anywhere else.


That's was just a few of my findings after a bit of poking around. Don't know if anyone will find them of any use.