Glo Triggering a library refresh automatically

[anarcat]

Hi,

I am working on a tool to automatically download content on the Kobo reader (I'm targeting the Glo HD because that's what I have, but it should work on other platforms as well). The tool is called Wallabako synchronizes wallabag entries into the kobo and generally works well.

The problem I have now is that the Kobo reader doesn't notice when new books are added. I have found a few different ways to work around the issue, but everything I found requires the user to tap the screen. For example, the Kobo-Mini-Wireless-Sync update.sh script does the following:

#!/bin/sh
echo usb plug add >>/tmp/nickel-hardware-status  # Simulate plugging in the usb
sleep 12 # Wait 12 sec for user to hit connect, and for any processes to run 
echo usb plug remove >>/tmp/nickel-hardware-status # Simulate the disconnect of our simulated usb, takes a few sec after the sleep for the change to show up on the device
# When the kobo detects USB connection, it seems to disable wifi, so only run the update after you are finished with uploading/downloading data

So basically, pretend something is plugged in, wait for the user to tap "Connect", then pretend-remove it. This works, but requires someone to tap the screen. If no one else is there, it won't work.

Other sites, for example this tutorial say that only the last line is necessary... It even suggests using /usr/local/Kobo/udev/usb with which i basically wiped the library (connecting through USB fixed that issue though).

There's also the refresh_library.sh trick from kobo-wget-sync that is basically to have a "virtual SD card", open and close it and notify Nickel that it's plugged in:

echo sd add /dev/mmcblk1p1 >> /tmp/nickel-hardware-status

The above command does nothing here. I also tried /dev/mmcblk1p3, the actual device that is on /mnt/onboard, but I suspect I am missing a key component of how the script works.

Finally, there's the Hacking SD card file system discussion which mentions all sorts of hacks on the sdcard, but it would require more research to see how we could use this as well.

I have also tried just calling the reboot command, but that doesn't trigger a refresh either.

Other solutions that were suggested in this thread:

• Write directly to the Kobo SQLite database: considered out of scope and too risky - the format can change under us and we don't want to reimplement the builtin scanner
• Simulate taps: would need to take a screenshot to figure out where the button is (yes, not always at the same spot!) and then figure out how to send taps to the input devices, see this discussion for more information
• Use the web browser: already suggested by the Chabotsi tutorial, takes over the web browser and downloads only one book at a time, not a solution...

So. What's the proper, simple way to trigger a refresh programmatically, without restarting the kobo and without user intervention?

Thanks!

[kido.resuri]

Currently, as far as I know, the only way for nickel to add new books is to connect to usb, safely disconnect, then nickel processes the content. Or maybe you have to "hack" the Kobo database, which has the potential risk of corrupting it and loosing your library. I'm also working on a project that can send content (books) to the Kobo over wifi, I'm trying to hack the built-in Sync and the Bookstore, but that also requires the user to interact with Kobo to get the content. Which is not really bad I think.

[davidfor]

You can download a book using the browser. This will add the book to the database.

[frostschutz]

You can simulate the tap too, only problem is, the button appears at different coordinate for every device so you have to provide a config file where users can set the coordinate that works unless you want to somehow detect it by looking at the frame buffer itself (I attempted such a thing in my screensaver mod with scanline but it's not a great solution).

Or you could let the user tap the first time and then just replay the same tap in the future, that would work until a firmware update slightly moves the dialog. Then you still have to deal with this popup appearing only after some unknown delay...

It would be nice to find a simpler way, but I don't know of one. I think even the browser just runs an update for the specific book and not search for other ones.

[anarcat]

Originally Posted by frostschutz

You can simulate the tap too, only problem is, the button appears at different coordinate for every device so you have to provide a config file where users can set the coordinate that works unless you want to somehow detect it by looking at the frame buffer itself (I attempted such a thing in my screensaver mod with scanline but it's not a great solution).

Or you could let the user tap the first time and then just replay the same tap in the future, that would work until a firmware update slightly moves the dialog. Then you still have to deal with this popup appearing only after some unknown delay...

It would be nice to find a simpler way, but I don't know of one. I think even the browser just runs an update for the specific book and not search for other ones.

That's really interesting. I know how I can take screenshots - but how do you simulate taps? That would seem to be a key component here...

Originally Posted by davidfor

You can download a book using the browser. This will add the book to the database.

That is not an option: I am running a program in the background and have no way to trigger the web browser. This is the approach taken by the Chabotsi tutorial, but only as a last resort. I don't like it so much: it means you can't actually use the browser for anything else anymore...

Originally Posted by kido.resuri

Currently, as far as I know, the only way for nickel to add new books is to connect to usb, safely disconnect, then nickel processes the content.

That is what I observed as well, but I was hoping people would prove me wrong.

Originally Posted by kido.resuri

Or maybe you have to "hack" the Kobo database, which has the potential risk of corrupting it and loosing your library. I'm also working on a project that can send content (books) to the Kobo over wifi, I'm trying to hack the built-in Sync and the Bookstore, but that also requires the user to interact with Kobo to get the content. Which is not really bad I think.

Yeah, I'm not going to touch the DB directly, except maybe to read metadata, but never write. Too risky and I don't want to rewrite the whole Kobo software, just add to it.

So far I'm using the "wait for tap" approach. It's not so bad, because the process is actually interactive: the user needs to cycle the Wifi connection to trigger the sync anyways, so it's expected that *something* will show up..

Keep the suggestions coming, I'll update the OP with changes...

[anarcat]

note: i ended up making a new thread about wallabako, but the conversation about triggering syncs should of course continue here. thanks!

[Ken Maltby]

"Tshering" used to have a fmon script to simulate a USB connection and disconnect, that would make for a scan, to update the DB. You could check out how he did it for KSM.

Luck;
Ken

[frostschutz]

Originally Posted by anarcat

That's really interesting. I know how I can take screenshots - but how do you simulate taps? That would seem to be a key component here...

Either a program that talks touchscreen or recording them, I described it a little over here https://www.mobileread.com/forums/sh...58#post3350658

[anarcat]

Originally Posted by Ken Maltby

"Tshering" used to have a fmon script to simulate a USB connection and disconnect, that would make for a scan, to update the DB. You could check out how he did it for KSM.

Luck;
Ken

that sounds like what the update.sh script does - does it require a user tap?

[kido.resuri]

yes, it does

[tshering]

Originally Posted by anarcat

that sounds like what the update.sh script does - does it require a user tap?

Yes, it does.
I remember faintly that I have seen another approach some years ago somewhere at this forum. I cannot remember the details, but it was about creating a virtual sd and downloading the books to this. If you mount the virtual sd nickel reads its contents and updates its library.

[anarcat]

Originally Posted by tshering

Yes, it does.
I remember faintly that I have seen another approach some years ago somewhere at this forum. I cannot remember the details, but it was about creating a virtual sd and downloading the books to this. If you mount the virtual sd nickel reads its contents and updates its library.

that sounds a *lot* like the refresh-library.sh script from the wget-sync hooks... but i couldn't make head or tails of that thing...

[frostschutz]

if you use the virtual sd you are stuck with it though. library entries for sd card are different from those in internal memory. Of course I guess you could work with the database from there and remove the sdcard binding but... from my experience with autoshelf, nickel caches a lot of data and thus does not see the database changes until reboot. it's only, again, actual usb connection when it re-reads the database and thus my autoshelf mod also requires one additional connect-to-pc step :-/

Someone bribe a Kobo developer, give us an API for things

[kido.resuri]

Originally Posted by frostschutz

from my experience with autoshelf, nickel caches a lot of data and thus does not see the database changes until reboot.

That's bad news for my project also

[helour]

Originally Posted by frostschutz

Someone bribe a Kobo developer, give us an API for things

Look at gtalusan's source code here.
Maybe he can help.