Aura H2O JavaScript problem: mirror my phone's screen in browser

[jefferson]

I am trying to build an app to mirror my phone's screen in the ebook's browser.
The problem I am running in with my kobo and kindle that the browsers don't support websockets. I have also tryed to update the image via setInterval, but it's not working.
I am running out of ideas how I can do it. Is there a spec of Kobo/Kindle's browser Javascript Dom API somewhere?
Any ideas how to browser debugging? Other methods to show an updated picture on KOBO/Kindle?

[geek1011]

Originally Posted by jefferson

Other methods to show an updated picture on KOBO/Kindle?

I'd recommend using @NiLuJe's FBInk, possibly with @shermp's Go bindings.

[sherman]

If you want to stick with JavaScript, server sent events could potentially be used, as I know the Kobo browser supports that. Whether you send the image directly to the Kobo by base64 encoding it, or sending update info for the Kobo to fetch an image from a URL is up to you.

[NiLuJe]

The Kobo is probably going to have an *extremely* hard time dealing with anything image-related on its own at anything better than a snails pace, especially given the native resolution on the average midrange smartphone.

If you go this route, better have the source preprocess the image as much as possible, so that the Kobo ends up having to do almost zero work besides dumping that to the framebuffer verbatim (i.e., match the target pixel format, ideally after having switched to 8bpp on the Kobo to save some bandwidth).

(The examples in the Python FBInk bindings have some musings about that sort of things).

[jefferson]

Originally Posted by sherman

If you want to stick with JavaScript, server sent events could potentially be used, as I know the Kobo browser supports that. Whether you send the image directly to the Kobo by base64 encoding it, or sending update info for the Kobo to fetch an image from a URL is up to you.

Do you have any examples? I have tried Websockets for sending images, but the Kobo browser have ignored it o crashed. Kindle's browser seems not to have websockets or fetch.
My last workaround Idea was to use setInterval to trigger an update of an img element with webser serving the latest image at a url.
Unfortunately setInterval doesn't seem to work

[jefferson]

Originally Posted by NiLuJe

The Kobo is probably going to have an *extremely* hard time dealing with anything image-related on its own at anything better than a snails pace, especially given the native resolution on the average midrange smartphone.

If you go this route, better have the source preprocess the image as much as possible, so that the Kobo ends up having to do almost zero work besides dumping that to the framebuffer verbatim (i.e., match the target pixel format, ideally after having switched to 8bpp on the Kobo to save some bandwidth).

(The examples in the Python FBInk bindings have some musings about that sort of things).

The plan is to optimize the image before sending - optimize black and white, resolution and choose only the part of the screen I care to see and not the whole screen.
I will check out FBInk, do you have more recommendations for an optimal image format - 8bpp is already something I don't understand.
Do you think it's possible to fetch and show images with FBInk? What would be an easy proof of concept?

[sherman]

Originally Posted by jefferson

Do you have any examples? I have tried Websockets for sending images, but the Kobo browser have ignored it o crashed. Kindle's browser seems not to have websockets or fetch.
My last workaround Idea was to use setInterval to trigger an update of an img element with webser serving the latest image at a url.
Unfortunately setInterval doesn't seem to work

See https://developer.mozilla.org/en-US/...er-sent_events for info about Server Sent Events.

TLDR, it's super easy to setup, assuming you don't need bi-directional communications. Although, if needed, you could set up your own return channel if required.

I know the Kobo browser supports it, because I've used it.

[jefferson]

thanks! Looks like a great option!

[NiLuJe]

I was talking about the 'DIY' approach, i.e., stream raw image data in the proper pixel format.

For the browser approach, you probably actually *don't* want to do that, because Nickel runs @ 32bpp, so you'll probably want to send images encoded in a format that decodes to RGBA to avoid an extra Y8 <-> RGBA roundtrip on Qt's side on the device.

(The extra bandwidth needed by RGBA over Y8 is *usually* still faster than the extra processing power needed by the grayscale expansion to RGBA, but YMMV. Overhead is in a different place in each case, but it's still there one way or the other. Have to DIY to get rid of any overhead on that front ).

[jefferson]

Originally Posted by sherman

See https://developer.mozilla.org/en-US/...er-sent_events for info about Server Sent Events.

TLDR, it's super easy to setup, assuming you don't need bi-directional communications. Although, if needed, you could set up your own return channel if required.

I know the Kobo browser supports it, because I've used it.

I am currently testing sse, but it doesn't seem to work. It might be some bug on my site, but it works on my desktop browser. Do you maybe still have some code samples which have worked for you?

Edit: nevermind, it works - I was just having es6 syntax in my js

[sherman]

Originally Posted by jefferson

I am currently testing sse, but it doesn't seem to work. It might be some bug on my site, but it works on my desktop browser. Do you maybe still have some code samples which have worked for you?

Make sure you aren't using any modern javascript features. ECMAScript 2015 (ES6) is definitely a no go.

I've used the event field myself. Something like the following worked for me:

var evtSource = new EventSource("/messages");
evtSource.addEventListener("body", function (event) {
    // code goes here
});

Note, using let for declaring variables is a no-go. Possibly const as well. To be safe, I'd stick with var.