07-18-2015, 05:27 PM | #1 |
Connoisseur
Posts: 53
Karma: 11844
Join Date: Jun 2014
Location: All over the place...
Device: KOBO AuraHD and GLO
|
How to trigger a book update on the Kobo...
Hi,
From a hacking perspective; I've got scripts on my kobo which are automatically downloading/generating/translating ebooks while the Kobo is NOT plugged into USB , and putting them in the normal place on /mnt/onboard as if I had copied them from my linux PC over to the reader; However, not even rebooting will cause the Kobo reader to "discover" these new books and allow me to read them. I have to plug in the USB cord, mount the reader on my PC, and unmount it -- before the reader suddenly detects them. That stinks, as it means I have to do the hokey pokey, disable telnet -- unload modules -- etc. etc. etc. before I can make all that happen. How can I fool the reader into thinking I have plugged the USB cord in, mounted the reader, and unmounted it -- using a shell script / without physically doing any of it? Is there a simple way? |
07-18-2015, 05:53 PM | #2 |
Evangelist
Posts: 407
Karma: 314204
Join Date: Jan 2015
Device: bq Avant XL, Kobo Aura H2O, Onyx Boox M96C Plus
|
KSM by tshering have a script to simulate the plug and triggered base update. You can give it a try
That's the code: Code:
echo usb plug add >> /tmp/nickel-hardware-status sleep 5 echo usb plug remove >> /tmp/nickel-hardware-status Last edited by boriar; 07-18-2015 at 05:56 PM. |
Advert | |
|
07-19-2015, 04:42 AM | #3 |
Connoisseur
Posts: 53
Karma: 11844
Join Date: Jun 2014
Location: All over the place...
Device: KOBO AuraHD and GLO
|
Odd, I must be jinxed.
I upload a pdf onto the reader through telnet, and excute the script lines you show. The "computer" connection popup happens on the kobo, and refuses to go away. So I say connect... and then run the sleep and disconnect, but it still says connected to computer. So eventually, I just pull the usb cord out. I go to 'library' on the Kobo, and not only is the new pdf/book not there -- but it says 'library empty', and all my normal books are gone. So, I press "sync", and it goes through some motions and all my normal library DRM books reappear, but the pdf I uploaded (Python_Programming_eBook_reader.pdf) does not show up in my library. It's a normal unprotected wikimedia commons pdf, and I can view it on my PC without issues -- so it's not a DRM protection issue. What am I missing? I can plug in and unplug the USB cable, which is all the script is supposedly trying to emulate; but that doesn't make it find the book either. I've not noticed the Kobo readier doesn't findi new books unless I *mount* the kobo reader, and unmount it from my PC -- which I can't do while the telnet client is installed over USB. Is there a way to simulate 'mounting the reader ? or perhaps, running sqlite on the database in .the /mount/.kobo directory, and adding some kind of entry that lets it know the system needs to be scanned ? My external SD card, I know when I run CFDISK on it, and save the partition table -- that Kobo erroneously thinks it was ejected and reinserted, and will scan and mount the first partition again. So, I know that there is probably a way to fake a udev or dbus event -- which causes the reader to believe that /mnt/onboard was unmounted and remounted.... I can probably solve it in a few days, but I was just thinking that someone else has probably already solved this issue in the past, and asking might be quicker than the school of hard knocks.... Last edited by fastrobot; 07-19-2015 at 04:51 AM. |
07-19-2015, 05:11 AM | #4 |
Linux User
Posts: 2,279
Karma: 6123806
Join Date: Sep 2010
Location: Heidelberg, Germany
Device: none
|
It's possible that it won't work with a USB cable connected. Maybe you could strace to find out how nickel detects this and then do something about it. I telnet through wifi. But still it's only half a solution since you have to confirm on the touchscreen. Although that could have its advantages.
Another way is to emulate ext. SD but your books have to be on ext. SD (real or emulated) for that. This does not exist for /mnt/onboard since that card can't be taken out (and the int. cardreader has no media change detection, for my magic memory upgrade mod which changes internal media I had to cheat, but no way nickel would handle). Maybe you can convince nickel to run the database upgrade code path with a ld library preload script somehow, what the nightmode does; if there is a better way to trigger this from the outside, I don't know of it. > it says 'library empty', and all my normal books are gone. Connecting to PC must umount the filesystems on the Kobo; if you're using telnet and you have anything running still working off the mounted fs, you might see filesystem corruption due to failed umounts and such. Last edited by frostschutz; 07-19-2015 at 05:14 AM. |
07-19-2015, 05:48 AM | #5 |
Evangelist
Posts: 407
Karma: 314204
Join Date: Jan 2015
Device: bq Avant XL, Kobo Aura H2O, Onyx Boox M96C Plus
|
Well, I rarely use telnet and never when use that script. I use it with my sideload png and epub and worked like a charm. Can it be caused by telnet running?
You can try that one (not mine, author is tshering). That one is used to mount and unmount from KSM like it was nickel: Code:
#!/bin/sh versionfilename=/mnt/onboard/.kobo/version #PRODUCT_ID is already exported by other script VERSION=$(awk -F"," '{print $3}' $versionfilename) SN=$(awk -F"," '{print $1}' $versionfilename) MODULE_LOADED=`lsmod | grep -c g_file_storage` VENDOR_ID=0x2237 DEV="/dev/mmcblk1p1" if [ -e /dev/mmcblk1p1 ]; then LUNS=/dev/mmcblk0p3,/dev/mmcblk1p1 else LUNS=/dev/mmcblk0p3 fi PARAMS="vendor=$VENDOR_ID product=$PRODUCT_ID vendor_id=Kobo product_id=eReader-$VERSION SN=$SN" sync echo 3 > /proc/sys/vm/drop_caches #umount -l /mnt/onboard umount -l /mnt/sd /sbin/insmod /drivers/$PLATFORM/usb/gadget/arcotg_udc.ko sleep 2 /sbin/insmod /drivers/$PLATFORM/usb/gadget/g_file_storage.ko file=$LUNS stall=1 removable=1 $PARAMS sleep 1 |
Advert | |
|
07-19-2015, 07:41 AM | #6 |
Wizard
Posts: 3,489
Karma: 2914715
Join Date: Jun 2012
Device: kobo touch
|
This is how the pretend_usb_connect_and_unconnect script is supposed to work:
If nickel fails for some reason to read new files into the database (this can also happen if you use it the regular way), nickel might refuse to read them at further trials too. Touching those files helps (sometimes?). Last edited by tshering; 07-19-2015 at 08:17 AM. |
07-19-2015, 09:34 AM | #7 |
Wizard
Posts: 4,465
Karma: 6900052
Join Date: Dec 2009
Location: The Heart of Texas
Device: Boox Note2, AuraHD, PDA,
|
@tshering; I wonder if a "silent" version of that, at the very beginning of the KoboRoot.tgz to install the fmon icons, might make the install a little more "fool" proof? It seems that people are still having problems installing working fmon .pngs.
Luck; Ken |
07-19-2015, 11:47 AM | #8 |
Wizard
Posts: 3,489
Karma: 2914715
Join Date: Jun 2012
Device: kobo touch
|
Ken, I guess I am missing your point. Could you please elaborate a little on this. As for a "silent" version, I am not sure whether this could be easily done.
|
07-19-2015, 03:06 PM | #9 | |
Wizard
Posts: 4,465
Karma: 6900052
Join Date: Dec 2009
Location: The Heart of Texas
Device: Boox Note2, AuraHD, PDA,
|
Quote:
As long as we need to use a KoboRoot.tgz to complete the process of installing working icons anyway, and you have the ability to "trigger" the nickel processing, couldn't you properly place the .png files and trigger the nickel processing, before conducting the rest of the installation that the KoboRoot.tgz was originally doing. That way it could be done properly, for the user. Eliminating a potential user error. Luck; Ken |
|
07-19-2015, 05:38 PM | #10 | |
Connoisseur
Posts: 53
Karma: 11844
Join Date: Jun 2014
Location: All over the place...
Device: KOBO AuraHD and GLO
|
Quote:
Thank you so much for warning me -- I know I would have managed to waste hours on this otherwise. I will make sure to rename the pdf file to something else before trying again.... There's nothing worse than to a make a mistake once and have it stuck on a bug forever and not knowing about it. |
|
07-19-2015, 09:11 PM | #11 | |
Grand Sorcerer
Posts: 24,907
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
|
Quote:
There might be a timestamp involved. That would explain why a touch had the books processed. If so, I don't know where it is. There is a "LastSyncTime" in the config file, but I have always assumed that is for the server sync. "__SyncTime" in the content table is the time the book was processed, so it might just be the maximum value of that. Or it could be in memory or on the root partition somewhere. In any case, it shouldn't be hard to come up with a test case for this. One thing that hasn't been mentioned is that downloading a book with the browser will get it added to the library. From memory, the browser is running as a separate process, so there has to be some communications between it and nickel. If you can work out what that is, it might be usable. Looking at the strings file I produced from libnickel, I can see: Code:
void SyncFileSystemCommand::findFilesToParse(const QString&, QStringList&) void SyncFileSystemCommand::pruneSideLoadedFiles(QStringList&, const QList<QString>&, QVector<Volume>&) void SyncFileSystemCommand::parseFB2(const QString&) const void SyncFileSystemCommand::parseKepub(const QString&) const void SyncFileSystemCommand::parseTextDocument(const QString&) virtual void SyncFileSystemCommand::execute() Of course, you could always add the book directly to the database. You have to parse the metadata and ToC and build the rows, but it is possible. I wrote some code a long time ago to do it in the calibre driver, but was never happy with it, so decided to leave it to the device. I think I understand the database better now, and do have code to update the ToC for some books. And a last thought, as you have devices with external SD cards, you could put the books there and unmount/mount it somehow. Or fake it using a directory on the internal card. Someone did something like this for a google drive sync. I don't know if it still works or not. |
|
07-20-2015, 04:40 AM | #12 | |
Wizard
Posts: 3,489
Karma: 2914715
Join Date: Jun 2012
Device: kobo touch
|
Quote:
|
|
07-20-2015, 09:35 AM | #13 | |
Grand Sorcerer
Posts: 24,907
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
|
Quote:
In any case, I did a quick test. I put an epub in the .kobo directory so it wouldn't get processed. Then I telneted in and copied it to the root and then used touch to set the date to last year. Then a connect and disconnect and it was processed. A single test isn't conclusive, but it agrees with all the behaviour I have seen. |
|
07-26-2015, 03:22 AM | #14 | |
Connoisseur
Posts: 53
Karma: 11844
Join Date: Jun 2014
Location: All over the place...
Device: KOBO AuraHD and GLO
|
Quote:
The web browser is likely a safer approach.... I launched nickel with strace -t -o /mnt/home/ni.log /user/local/Kobo/nickel -platform kobo -skipFrontLoad to follow your advice. Nickel runs, and then I launched the web browser to get a random book; eg: http://azardi.infogridpacific.com/resources.html the "recollections of war" book at the top, and then said view it. And now the bad news: The first instance of "recoll" in the strace, is as a string already in memory: Code:
20:38:18 stat64("/mnt/onboard/recollections-of-wartime.epub", 0x7ea1e300) = -1 ENOENT (No such file or directory) 20:38:18 open("/mnt/onboard/recollections-of-wartime.epub", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE|O_CLOEXEC, 0666) = 26 20:38:18 fcntl64(26, F_SETFD, FD_CLOEXEC) = 0 20:38:18 brk(0x3066000) = 0x3066000 20:38:18 brk(0x30b2000) = 0x30b2000 20:38:18 mmap2(NULL, 1052672, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3b023000 20:38:18 mremap(0x3b023000, 1052672, 2101248, MREMAP_MAYMOVE) = 0x3b023000 20:38:18 mremap(0x3b023000, 2101248, 4198400, MREMAP_MAYMOVE) = 0x3b023000 20:38:18 mremap(0x3b023000, 4198400, 8392704, MREMAP_MAYMOVE) = 0x3b023000 20:38:18 mremap(0x3b023000, 8392704, 16781312, MREMAP_MAYMOVE) = 0x3b023000 20:38:18 mremap(0x3b023000, 16781312, 33558528, MREMAP_MAYMOVE) = 0x3b023000 20:38:18 brk(0x2392000) = 0x2392000 20:38:18 fstat64(26, {st_mode=S_IFREG|0755, st_size=0, ...}) = 0 20:38:18 write(26, "PK\3\4\24\0\0\0\0\0\362\n0Coa\253,\24\0\0\0\24\0\0\0\10\0\0\0mi"..., 29676824) = 29676824 20:38:18 munmap(0x3b023000, 33558528) = 0 20:38:18 close(26) = 0 I only see occasional netlink calls, and a continual polling of descriptors 6,11,12 via pselect6 before this point in the strace, but no reads from a file descriptor that supply a name.... So, I think there might be shared memory being used between nickel and the web-browser (I'm not very familiar with strace, but this looks like threaded operation or shared memory.) Since this is a trace of an open for 'write', then I assume we are seeing the web browser save the file via. nickel using shared memory. The next time the filename gets mentioned, it's being writev() to a file descriptor 35 which was never opened by nickel anywhere in the strace. Code:
20:39:13 writev(35, [{"l\1\0\1\32\0\0\0\f\0\0\0\177\0\0\0\1\1o\0\25\0\0\0/org/fre"..., 144}, {"\25\0\0\0com.kobo.adobe.Reader\0", 26}], 2) = 170 20:39:13 read(35, 0x163fa48, 2048) = -1 EAGAIN (Resource temporarily unavailable) 20:39:13 writev(35, [{"l\1\1\1\210\0\0\0\r\0\0\0\177\0\0\0\1\1o\0\25\0\0\0/org/fre"..., 144}, {"\203\0\0\0type='signal',sender='org.fr"..., 136}], 2) = 280 ... 20:39:14 writev(35, [{"l\1\0\0016\0\0\0\24\0\0\0g\0\0\0\1\1o\0\1\0\0\0/\0\0\0\0\0\0\0"..., 120}, {"1\0\0\0file:///mnt/onboard/recollec"..., 54}], 2) = 174 Unless the web browser's source code is in the respository, this looks pretty hopeless to me. It's doing it all internally, and not over kernel calls. To be able to do something like that, will probably require me to to write a plugin that talked to nickel and passed it a file-name via shared memory. I think there are some poker plugins for nickel in the repository, so there might be enough information to trick it into looking for an epub that was supposedly 'downloaded' even if it wasn't. But what little I have found on writing plugins for nickel, basically says it's nearly impossible because its so badly documented. But now the really good news: I can see that after I download that random epub book using the web server, that nickel suddenly found my thinkpython.epub book that it had been stuboornly ignoring up and until now, especially with the USB cable plugged in and telnet running. So, there is a fairly easy and reliable way to trick the kobo into finding new books; eg: Put a file on / called getbooks.html that looks like: Code:
<HTML> <A href="file:///refresh.epub">refresh my booklist now</A> </HTML> Code:
#!/bin/sh echo "<HTML><A href=\"file:///$1\">refresh my booklist now</A></HTML>" > /getbook.html #rm $1 # optionally remove original book as it will be copied to the /mnt/onboard Last edited by fastrobot; 07-26-2015 at 03:30 AM. |
|
07-26-2015, 04:31 AM | #15 |
Linux User
Posts: 2,279
Karma: 6123806
Join Date: Sep 2010
Location: Heidelberg, Germany
Device: none
|
This parsing and stuff is done by an instance of /usr/local/Kobo/adobehost(-launcher.sh), it has three modes (reader, parser, search). That's probably your fd 35. You might get a more complete picture if you kill adobehost first (nickel re-starts it as needed) and then strace -ff nickel.
Interesting idea with the browser. Modifying the database: dunno if it works for books, it does not for collections, nickel does not re-read until it leaves USB host mode. For the books table I know it changes rowIDs of books around a lot (basically everytime you open a book it gets a new rowID) and I don't know what assumptions nickel makes about those so... I'd rather not risk it. Last edited by frostschutz; 07-26-2015 at 04:36 AM. |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Eink Framebuffer Update Trigger | geekmaster | Kindle Developer's Corner | 36 | 12-28-2019 06:24 PM |
Finally pulled the trigger on a Kobo Aura HD | Rps | Kobo Reader | 17 | 08-31-2014 09:31 PM |
kobo, google play update, calibre update | tracym | Kobo Tablets | 16 | 06-25-2012 03:24 PM |
How can I update creation time for a title when I update the book info or convert it | setherd | Calibre | 3 | 10-30-2010 01:51 PM |
Problems with locked book (ADE) Kobo store after update | taming | Kobo Reader | 0 | 07-14-2010 04:05 AM |