Aura H2O Turn pages by voice

[tshering]

@Ecallan
I just thought you might perhaps have overlooked this part of the description in Windows 10 signed RNDIS driver for USBNetwork:

You should see a new network card Kindle USB RNDIS Device (USBNetwork enabled) in "Network and Sharing Center" > Change adapter settings. Assign a static IP such as 192.168.15.1 and you can ping the device (ping 192.168.15.244).

As for ping, the address of our Kobo device is 192.168.2.101 (and not 192.168.15.244).

I did not mention that for enabling USBnet in KSM you need to

• select "usb" > "usbnet toggle.sh"

But I think you know this already.

[Ecallan]

I did toggle usbnet on my kobo reader and it says USB netsupport enabled at the top of the screen.
In network sharing centre I have ethernet 6 – network 3 Kindle USB RNDIS (USBnetwork enabled)

I opened properties on Internet Protocol Version 4 (TCP/IPv4)

I set the static ip to 192.168.2.101
The submask to 255.255.255.0
The default gateway to 192.168.2.101
I've left the DNS server section blank

However of when I open the cmd screen and do ipconfig I get this:
Autoconfiguration IPv4 address 169.254.118.43
Subnet mask 255.255.0.0
Default Gateway 192.168.2.101

When I ping 192.168.2.101 I get time out and 100% loss

Sorry for not getting back to you until now. I would be with my computer until later this evening.

[Ecallan]

I'm sorry, that should say I won't be at my computer from now until later this evening because I have to go someplace. I'll be back later this evening.

[tshering]

192.168.2.101 is the ip of the Kobo device. For the adapter on the pc you have to take a different one. The post "Windows 10 signed RNDIS driver for USBNetwork" proposes to "assign a static IP such as 192.168.15.1." This is for the kindle, for the kobo this might to translate to "assign a static IP such as 192.168.2.1." I think I have 192.168.2.100, but I am not certain about this (I am again in the office). Anyway I would try this one or some other that starts with "192.168.2.", except of 192.168.2.101.

[Ecallan]

Success

I've set a static ip address as 192.168.2.1 and nothing for default gateway
I pinged 192.168.2.101 and got 4 packets sent & 4 packets received 0% loss

[tshering]

This is great. You are ready! I however was unable to finish the test package today. I hope I have a chance to finish it tomorrow. The package will overwrite your KSM configuration file. Did you do any changes that you want to keep (apart from select "tools" > "activate" > "set runmenu settings.msh" > "always")?
We did not yet talk about which version of the Kobo firmware you are running. The important point is: Do you have a version with the new home screen?

[Ecallan]

No rush on finishing the package.

No I didn't make any other changes. I did nickel diverse, whatever that does.

I checked in settings & device info, the software version is 4-5-9587 (c77383d7d1 7/13/17)

[tshering]

Originally Posted by Ecallan

No rush on finishing the package.

I will continue on this this evening.

Originally Posted by Ecallan

No I didn't make any other changes. I did nickel diverse, whatever that does.

nickel diverse did only open a sub menu that gives you some options. So you did not do any change be simply opening it.

[tshering]

Here is the first test package.

Installation:

• copy KoboRoot.tgz to .kobo, safely eject and disconnect the reader from the pc.
• in KSM (home) select: handle update > install update

Usage:
As it was already before the installation of this package, you have around 20 seconds to select an entry of the main menu. If you do not select anything, nickel will be started automatically (without remote control enabled).

• enable USBnet
• select "rmc start"
• on your PC open MS internet explorer and copy the following line into the location bar (and then press enter):
• http://192.168.2.101:8087/rmcnickelvoice.html
• there should now open a html page with a lot of buttons, if not select "rmc stop," disable USBnet and tell me about it.
• if you see the page, select on the Kobo device "start nickel" and let nickel completely load, then you can start testing the remote control.

In the following I give an example of what to do with the different available commands. I put the commands between quotation marks. "next page" means press the button "next page" with the mouse or by voice. (One has to choose a zoom level so that all buttons are visible. One can only reach visible buttons with the voice.)

Spoiler:

Let us open a book that is listed in the first row of the home screen:

• "home open book one"

Do this only when on this position, there is a real book. If it is exit_nickel.png or so, then it is not a good idea to open it now.

When the book is open, try

• "next page" and "previous page"

Let us open a book from the library. Do one step after the other, as you would when realy touching the screen:

• "open top bar"
• "open main menu"
• "open main menu my books"

you can page through the library by

• "next page" and "previous page"

you can select a book from the current page:

• "library open book one" or two/three/four/five

Let us return to the home:

• "open top bar"
• "open main menu"
• "open main menu home"

Let us go back to KSM:

• "exit nickel"

This need some seconds before you see any action. Note: You have always to return to the home screen, before you say "exit nickel." If not, nickel will not make the necessary changes to the data base, as for instance the last position in the book. (This is at least my guess. I did not try to "exit" from any other position than the home, and would never try to "exit" with a book opened.)

In the beginning I had difficulties to make the Windows Speech Recognition understand what I say. But now it works well. I am not sure whether the Speech Recognition learned to get used to my pronunciation, or whether my pronunciation improved. I add the file rmcnickelvoice.html to the zip file. You can open it on your pc and test how to select the buttons without having the reader connected to the pc.

I should maybe mention that you have to disable USBnet before you can connect the device as a storage drive to your pc.

General remark: This package is for Kobo Aura H2O. KSM 08 must already be installed.

As always USE AT YOUR OWN RISK!

[Ecallan]

I did as directed and I'm very pleased to say that it works a treat. So your long hours of work on this project have really paid off! It is simply fantastic to now be able to turn pages by voice. Thank you for the additional commands, but really the most important one is "next page", followed closely by number two "previous page".

As a follow-on from your work and the trial, I'm going to look into getting an Intel stick PC, coupled with a battery pack and the microphone. I should be able to set it up that the PC starts up and opens the HTML page that you've constructed. I should also be able to get it to load the dictation software in the start-up process. I've done this at work with a relatively slow PC.

So I sincerely thank you for persevering with this project. Hopefully others will come across this when searching for a turn pages by voice solution. You might consider removing the earlier exchanges when we were trying a different solution.

You might also be interested in following this link were another person built a tailor-made solution
http://hackaday.com/2015/05/19/turni...th-your-voice/
What do you think?

Best regards

Eugene

[tshering]

Originally Posted by Ecallan

I did as directed and I'm very pleased to say that it works a treat. So your long hours of work on this project have really paid off! It is simply fantastic to now be able to turn pages by voice. Thank you for the additional commands, but really the most important one is "next page", followed closely by number two "previous page".

I am glad that it works and that you like it.

Originally Posted by Ecallan

As a follow-on from your work and the trial, I'm going to look into getting an Intel stick PC, coupled with a battery pack and the microphone. I should be able to set it up that the PC starts up and opens the HTML page that you've constructed. I should also be able to get it to load the dictation software in the start-up process. I've done this at work with a relatively slow PC.

I would not rush into things. An intel stick PC seems a rather expensive choice for two buttons (next page, previous page), and maybe also a heavy battery consumer. There is possibly some (development) board that can do the needed job (sending requests over usbnet) and is compatible with some speech recognition shield. I simply have no idea about this kind of stuff.

If you go the Intel stick PC way, then there are still some things to be aware of. There are some problems that I saw with the Windows speech recognition when testing.

It responds to any sound whether it is produced by putting a tea cup on the table, closing a door, what people say passing by, etc. If it cannot make any sense of the sounds, it simply asks "What was this?" So this is not so much a problem. But if it can somehow identify it as kind of command, things are different.

It happened to me that it opened another tab of the browser in search of the unknown command. If you do not have a screen, you simply do not know what is happening. And if you figure it out, it might be difficult with speech commands to put the focus back on the tab of the browser that you need (I did not try this). Once it interpreted what it heard as a command to start a certain batch file on my pc. Fortunately, this batch file only sets some environment variables and waits for further commands. But I have certainly batch files on my pc that could do some harm. Without seeing the screen it is difficult to know what is happening on the pc.

I guess I want to say that you need some way to restrict the range of actions the speech recognition is allowed to do. I did not look into this.

Originally Posted by Ecallan

You might also be interested in following this link were another person built a tailor-made solution
http://hackaday.com/2015/05/19/turni...th-your-voice/
What do you think?

It is a nice project. Maybe you should consider buying a kindle after all.

Best wishes,
tshering

[Ecallan]

Originally Posted by tshering

I am glad that it works and that you like it.

I would not rush into things. An intel stick PC seems a rather expensive choice for two buttons (next page, previous page), and maybe also a heavy battery consumer. If you go the Intel stick PC way, then there are still some things to be aware of. There are some problems that I saw with the Windows speech recognition when testing.
[tshering

Yes I've considered these points and will not make any rash decisions. For now, I'm very happy to be able to read at home by voice alone.

Thanks again

Eugene

[Susandrea]

Hi Tshering,
I've just been reading a four-year-old thread about voice activation for Kobo readers. I'm look for a solution for a friend who is paralyzed from the neck down. He was an avid reader before his accident and finds tablet reading too cumbersome, among other things because of the glare. Have you developed your solution any further? Are you aware of any other solutions that may have come into the market in the meantime?
Hope you see this post!
All the best,
susan

[davidfor]

Originally Posted by Susandrea

Hi Tshering,
I've just been reading a four-year-old thread about voice activation for Kobo readers. I'm look for a solution for a friend who is paralyzed from the neck down. He was an avid reader before his accident and finds tablet reading too cumbersome, among other things because of the glare. Have you developed your solution any further? Are you aware of any other solutions that may have come into the market in the meantime?

I don't know if @Tshering has done anything on this recently. He is not around much now, so he might not see this for a while. I don't know if the solution above still works, but things have changed so there might be alternatives.

Firstly, if the only issue with tablets is the screen, there are Android based tablets with e-ink screens. They may be able to run the apps that your friend needs to use.

It might also be worth looking at the Kobo Elipsa. This has Bluetooth support for remote controls and keyboards. There may be something available to help with using the device.