11-28-2012, 02:24 AM | #211 |
Grand Sorcerer
Posts: 24,907
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
|
|
12-02-2012, 09:13 AM | #212 |
Member
Posts: 15
Karma: 250
Join Date: Nov 2012
Device: Kobo Glo
|
I'd like to use this plugin to add a custom download option to the 'Tools' menu on the home screen. I guess this would involve using executeSystemCmd to run a shell script on the device which would (ideally) download from a remote HTTP server a text file containing a list of file URLs to be downloaded, then download each file individually.
On a Linux-based computer this is a relatively easy task, using a command line tool like 'wget' to download the remote files, but on the Kobo I'm not sure what command to use to retrieve remote URLs. Does anyone know if there is any way to do this by running such a command on the device? |
Advert | |
|
12-02-2012, 09:55 AM | #213 |
Enthusiast
Posts: 39
Karma: 454
Join Date: Jul 2012
Location: Brussels
Device: Kobo Touch
|
The Kobo is also running a Linux-based OS and has the wget command available. Thus, you can implement your script like on any other Linux box.
|
12-03-2012, 08:40 AM | #214 | |
Member
Posts: 15
Karma: 250
Join Date: Nov 2012
Device: Kobo Glo
|
Quote:
Code:
Plugin::executeSystemCmd("/bin/wget URL") Also, where will this downloaded file be saved? I guess I want to save it to the root directory (how do I refer to this internally for a system command?), but then, will the Glo notice that a new file has appeared there and needs to be added to my library, or do I have to manually call some kind of library update? |
|
12-04-2012, 12:19 PM | #216 |
Member
Posts: 15
Karma: 250
Join Date: Nov 2012
Device: Kobo Glo
|
Custom Sync - working, but clunky
Okay, I've got a successful custom sync working, at least as a proof-of-concept version. It currently requires you to have access to your own web server, running PHP (I'm running this on my own Linux desktop) - this is because we need to generate a list of file URLS for the script on the Kobo to retrieve. If anyone knows how to e.g. get the built-in Calibre HTTP server to generate a plain text file of new files in the last X days, that would be even better. This has been tested only on the Kobo Glo with firmware 2.1.5.
Here's how it works: Add to .kobo/koboplugins.ini: Code:
CSync=Plugin::connectWifi()>>Plugin::executeSystemCmd("/mnt/onboard/run.sh &") Now create the following scripts in the Kobo root folder (i.e. the root folder when you access it as a USB drive): run.sh Code:
#!/bin/sh touch /mnt/onboard/started_sync nohup /mnt/onboard/sync.sh & sync.sh Code:
#!/bin/sh # Add your HTTP server details here LIST_URL="http://192.168.0.123/books/filelist.php"; PING_IP="192.168.0.123" ATTEMPTS=30 #Uncomment the next line to always download files in the list (i.e. ignore existing downloaded file, save with unique name) # NOTE: for testing only! Will create duplicate books on your device #ALWAYS_DL=1 # wait for wifi network to come online; give up after $ATTEMPTS tries tries=1 while ! ping -c 1 -q $PING_IP; do if [ "$tries" -gt "$ATTEMPTS" ] ; then timestamp=`date` echo "sync: aborted after $ATTEMPTS connection attempts ($timestamp)" >>/mnt/onboard/sync.log break; fi sleep 2 tries=$(( $tries + 1 )) done successes=0 LIST=`/usr/bin/wget -q -O - $LIST_URL` IFS=$'\n' # make the for loop break up the list on newlines for url in $LIST; do filename=`echo $url|sed -e 's|.*/||g'` if [ $ALWAYS_DL ] ; then timestamp=`date +%Y%m%d-%k%M%S` if /usr/bin/wget "$url" -O "/mnt/onboard/$timestamp-$filename" >>/mnt/onboard/wget.log 2>&1 ; then successes=$(( $successes + 1 )) echo "Downloaded (forced) $url to $timestamp-$filename" >>/mnt/onboard/sync.log else timestamp=`date` echo "Failed to force download $url ($timestamp)" >>/mnt/onboard/sync.log fi else if /usr/bin/wget "$url" -O "/mnt/onboard/$filename" >>/mnt/onboard/wget.log 2>&1 ; then successes=$(( $successes + 1 )) echo "Downloaded $url" >>/mnt/onboard/sync.log else timestamp=`date` echo "Failed to download $url (file exists? $timestamp)" >>/mnt/onboard/sync.log fi fi done if [ "$successes" -gt "0" ] ; then timestamp=`date` echo "Downloaded $successes files ($timestamp)" >>/mnt/onboard/sync.log nohup /mnt/onboard/reindex.sh & fi reindex.sh Code:
#!/bin/sh echo usb plug add >> /tmp/nickel-hardware-status sleep 10 echo usb plug remove >> /tmp/nickel-hardware-status These files all need to be set as executable, i.e. "chmod 755". Here is the PHP script which will create the expected list of file URLs to download; the epub files must in the same directory as the script. Code:
<?php $path = substr($_SERVER[PHP_SELF], 0, strrpos($_SERVER[PHP_SELF], '/')); $find = 'find . -iname \\*.epub'; //-mtime 0 '; $ar = array(); exec($find, $ar); foreach($ar as $file){ $url = "http://" . $_SERVER['SERVER_NAME'] . $path . '/' . str_replace(' ', '%20', substr($file, 2)); print "$url\n"; //print "<a href=\"$url\">$url</a><br>"; } ?> Whew. Ok, once all that is in place, you should be able to automatically download files to your Kobo and have them automatically indexed (after the fake USB-connect sequence). The script will not re-download files that it has already saved to the Kobo unless you uncomment the ALWAYS_DL line in sync.sh, but note that this creates unique filenames each time it downloads a file, so you'll get multiple versions of the same book in your library. The script does some logging to wget.log and sync.log in the same root directory as the scripts themselves. TODO: - Less interactive way to force checking/reindexing of new files. Anyone know if this can be done via a shell command? - Neater way to make sure wifi is available before doing download - Fix label bug that causes menu item to appear as "WiFi" - Ideally it would be good to either serve the files from the Calibre server, if it can be made to generate a simple enough list of URLs of new files. Alternately, it would be good to be able to remotely host the files so they can be synced even when your own computer is off, i.e. Dropbox or similar; however with Dropbox there's no way to generate the list of file URLs, and no way to list the contents of a web folder to get all the files. If anyone can think of a generic alternative that would work in this case, that would be good. - Text updates at the top of screen - currently it gets stuck on "Connecting", I think this is the Wifi command. Not sure how to change the this text. - A way to extract emailed articles and save them as epub files, so they can be downloaded using this process, i.e. make use of browser extensions that allow you to "email this page to your kindle"... any ideas? Attached is a zip file with the above scripts that need to be extracted into the root folder, and the PHP script for your server. The Kobo shell scripts will probably still need to have the correct execute permissions set - this will be hard if you haven't enabled telnet for the device. Let me know how this works if you try it out. Last edited by textchimp; 12-04-2012 at 08:30 PM. |
12-04-2012, 02:37 PM | #217 |
Addict
Posts: 369
Karma: 37869
Join Date: Sep 2011
Device: Kobo eReader Touch, Kobo Aura HD
|
Great work !
If you want I can add a special page in Cops (http://blog.slucas.fr/en/oss/calibre-opds-php-server) that produce the correct file format. It's been a while since I've used the plugin, but you had a goo idea. |
12-05-2012, 02:06 PM | #218 |
Junior Member
Posts: 4
Karma: 10
Join Date: Dec 2012
Device: Kobo Glo
|
Got a Kobo Glo last weekend. Running firmware 2.1.5. People on this thread seem to confirm that the plugin interface works on the Glo with this firmware.
I followed the instructions - connected my Glo via USB, unzipped both files to the .kobo directory, and ejected the device from my PC. I think the .tgz file is being read, because when I do this, the screen says that the software is updating. My issue is that after that, the old stock interface is still there, with no changes. I open a book, and close it. Still no change. I shut down the Kobo, boot it back up, open and close a book, and still nothing. I've tried this with my own *.ini content, and with an unmodified *.ini file downloaded from the first post of this thread. Neither has worked. What am I doing wrong? |
12-06-2012, 09:10 AM | #219 |
Member
Posts: 15
Karma: 250
Join Date: Nov 2012
Device: Kobo Glo
|
Does anyone know if there's a command available on the embedded Linux running on the Kobo Touch/Glo which could send a file to a remote host? Ideally something like an automated FTP file send (i.e. ncftpput - but clearly that's not available), although even making an HTTP POST would probably be enough.
I'm trying to write a script to send all the annotations/highlights on the device to a remote server. |
12-06-2012, 09:27 AM | #220 | |
Connoisseur
Posts: 62
Karma: 31414
Join Date: Sep 2012
Location: France
Device: Kobo Glo
|
Quote:
Quite widespread and basic command on Linux systems. |
|
12-06-2012, 02:30 PM | #221 |
Enthusiast
Posts: 49
Karma: 208
Join Date: Nov 2012
Device: Kobo Glo
|
Do we have curl access to our Kobo's?
Then we could use this: http://superuser.com/questions/86043...part-form-data (Second post) I was thinking to create some sort script that synces epubs booksmarks (not the offical kobo books) with my android smartphone. Does anyone know where the Kobo saves the bookmarks? In the sqllite database I guess? |
12-06-2012, 04:24 PM | #222 |
Member
Posts: 15
Karma: 250
Join Date: Nov 2012
Device: Kobo Glo
|
Sending highlights/annotations to remote server
Ha, ok yes, wget again. I've got this to work, again as a proof-of-concept, but the version of wget on the Kobo is limited - it does not have the --post-file=FILE command line option, but it does have (undocumented) the --post-data="DATA" option. This means you can send some post data, but only via the command line, and it seems to limit how much data you can send. (Anyone got a better idea?)
Anyway, here's a working example to grab all the annotation data from /mnt/onboard/Digital\ Editions/Annotations and post it to a PHP script on a server. First create a menu entry in koboplugins.php like: Code:
CSync=Plugin::connectWifi()>>Plugin::executeSystemCmd("/mnt/onboard/run-notesync.sh &") run-notesync.sh: Code:
#!/bin/sh touch /mnt/onboard/started_notesync nohup /mnt/onboard/notesync.sh & Code:
#!/bin/sh ATTEMPTS=40 # change the addresses below for your server REMOTE_URL="http://serveraddress/path-to/post.php" PING_ADDRESS="serveraddress" POSTDATA="" # wait for wifi network to come online; give up after $ATTEMPTS tries tries=1 while ! ping -c 1 -q $PING_ADDRESS; do if [ "$tries" -gt "$ATTEMPTS" ] ; then timestamp=`date` echo "sync: aborted after $ATTEMPTS connection attempts ($timestamp)" >>/mnt/onboard/notesync.log exit; fi sleep 2 tries=$(( $tries + 1 )) done POSTDATA="" FILES=`find "/mnt/onboard/Digital Editions/Annotations/" -iname \*.annot -exec grep -l '<text>' {} \;` IFS=$'\n' for file in $FILES; do POSTDATA=${POSTDATA}`awk '/<dc:title>/ || /<dc:creator>/ || /<text>/ || /<\/text>/' "$file"` POSTDATA="${POSTDATA}\n\n" done #echo -e "$POSTDATA" wget $REMOTE_URL -O /mnt/onboard/notesync.log --post-data="content=$POSTDATA" timestamp=`date` echo -e "\n\nSent on $timestamp \n" >>/mnt/onboard/notesync.log (NOTE: it looks like the way the notes are saved in these .annot files ignores paragraph breaks - it just runs text from different paragraphs together. Annoying! In any case, the awk filtering done in the script will ignore lines without a <text> or </text> tag; needs a better multiline regex...) On the server end, you need a simple PHP script like this: post.php Code:
<?php if(isset($_POST['content'])) { file_put_contents('notes.html', $_POST['content']); exit; } ?> Last edited by textchimp; 12-06-2012 at 09:25 PM. |
12-06-2012, 05:41 PM | #223 | |
Grand Sorcerer
Posts: 24,907
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
|
Quote:
The annotations and highlights are in the annot file that textchimp is using. These are also in a database table called "Bookmark". Hopefully any ADE based reader could reuse the file. Also, the file on the Kobo's takes precedence. If you delete the records in the table, the annotations are loaded from the file. If you delete the file, the annotations are removed. The current place in the book is stored in the "content" table. Each book has a row with ContentType=6. Books that are being read have a value in the column "ChapterIDBookmarked". This has a reference to the current chapter. The reference is the key to the content table for a ContentType=9 row. This row has a value in the column "adobe_location". This is the current location in the chapter. The following SQL will show the current position for all sideloaded books: Code:
SELECT c1.Title, c1.Attribution, c1.DateLastRead, c1.ChapterIDBookmarked, c1.ReadStatus, c1.___PercentRead, c2.ContentId, c2.Title, c2.adobe_location FROM content c1 join content c2 on c1.ChapterIDBookmarked = c2.ContentID where c2.bookid like 'file:///mnt/onboard%' |
|
12-06-2012, 07:07 PM | #224 |
Junior Member
Posts: 7
Karma: 10
Join Date: Dec 2012
Location: Canada
Device: Kobo Touch
|
Just wanted to say many, many thanks for this plugin. It's making my Kobo experience so much better. :-) Love the enhanced reading footer especially.
|
12-07-2012, 02:57 AM | #225 |
Member
Posts: 17
Karma: 11628
Join Date: Oct 2012
Device: Kobo Touch
|
textchimp: I'm also using the same approach to "syncing" books. You may want to use the plugin's command chaining and its sync mechanism instead of simulating a usb connecting. Also, the busybox wget will not work with secure connections. So you may want to use curl instead.
I'm currently also using a simple php script with a simple interface to add URLs that I want the kobo to download. When I get some free time, I hope to do Dropbox syncing using their REST api. |
Tags |
kobo, plugin |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Where can i get firmware 2.2.1 alpha for pocketbook 912 | johannes866 | PocketBook | 1 | 02-13-2012 04:57 AM |
Firmware 2.0.3 Pocketbook IQ - another firmware is release for IQ | tvpupsik | PocketBook | 26 | 12-13-2010 03:03 PM |
DR1000 Chinese Vertical Layout uds plugin plaintext Release | spuggy | iRex | 4 | 09-18-2010 02:00 PM |
dotReader alpha release next month with future iLiad support possible | Alexander Turcic | News | 7 | 09-28-2006 12:24 PM |
OpenBerg Milestone Release 1 alpha released | Colin Dunstan | Reading and Management | 0 | 07-13-2005 07:41 AM |