Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Readers > Kobo Reader > Kobo Developer's Corner

Notices

Reply
 
Thread Tools Search this Thread
Old 11-28-2012, 03:24 AM   #211
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 6,090
Karma: 6238033
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo Touch, Kobo Glo
Quote:
Originally Posted by textchimp View Post
Anything in particular? I'm setting margin and padding already...
I mean that the paragraphs may be using a specific style rather than just the default for "p". Check the html files for styles on the paragraphs.
davidfor is offline   Reply With Quote
Old 12-02-2012, 10:13 AM   #212
textchimp
Member
textchimp has a complete set of Star Wars action figures.textchimp has a complete set of Star Wars action figures.textchimp has a complete set of Star Wars action figures.
 
Posts: 15
Karma: 250
Join Date: Nov 2012
Device: Kobo Glo
I'd like to use this plugin to add a custom download option to the 'Tools' menu on the home screen. I guess this would involve using executeSystemCmd to run a shell script on the device which would (ideally) download from a remote HTTP server a text file containing a list of file URLs to be downloaded, then download each file individually.

On a Linux-based computer this is a relatively easy task, using a command line tool like 'wget' to download the remote files, but on the Kobo I'm not sure what command to use to retrieve remote URLs. Does anyone know if there is any way to do this by running such a command on the device?
textchimp is offline   Reply With Quote
Old 12-02-2012, 10:55 AM   #213
sven
Enthusiast
sven has a complete set of Star Wars action figures.sven has a complete set of Star Wars action figures.sven has a complete set of Star Wars action figures.sven has a complete set of Star Wars action figures.sven has a complete set of Star Wars action figures.
 
Posts: 39
Karma: 454
Join Date: Jul 2012
Location: Brussels
Device: Kobo Touch
The Kobo is also running a Linux-based OS and has the wget command available. Thus, you can implement your script like on any other Linux box.
sven is offline   Reply With Quote
Old 12-03-2012, 09:40 AM   #214
textchimp
Member
textchimp has a complete set of Star Wars action figures.textchimp has a complete set of Star Wars action figures.textchimp has a complete set of Star Wars action figures.
 
Posts: 15
Karma: 250
Join Date: Nov 2012
Device: Kobo Glo
Quote:
Originally Posted by sven View Post
The Kobo is also running a Linux-based OS and has the wget command available. Thus, you can implement your script like on any other Linux box.
Ah, good to know! However I've just tried to run wget without apparent success. Should something like

Code:
Plugin::executeSystemCmd("/bin/wget URL")
do what I want? Will wget be in /bin ? Is there some way to log output from these commands?

Also, where will this downloaded file be saved? I guess I want to save it to the root directory (how do I refer to this internally for a system command?), but then, will the Glo notice that a new file has appeared there and needs to be added to my library, or do I have to manually call some kind of library update?
textchimp is offline   Reply With Quote
Old 12-03-2012, 01:42 PM   #215
Koboyashi
Zealot
Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.Koboyashi ought to be getting tired of karma fortunes by now.
 
Posts: 135
Karma: 207434
Join Date: Dec 2011
Device: Kobo Touch & Glo
Try /usr/bin/wget - and here's a man page of wget.
Koboyashi is offline   Reply With Quote
Old 12-04-2012, 01:19 PM   #216
textchimp
Member
textchimp has a complete set of Star Wars action figures.textchimp has a complete set of Star Wars action figures.textchimp has a complete set of Star Wars action figures.
 
Posts: 15
Karma: 250
Join Date: Nov 2012
Device: Kobo Glo
Custom Sync - working, but clunky

Okay, I've got a successful custom sync working, at least as a proof-of-concept version. It currently requires you to have access to your own web server, running PHP (I'm running this on my own Linux desktop) - this is because we need to generate a list of file URLS for the script on the Kobo to retrieve. If anyone knows how to e.g. get the built-in Calibre HTTP server to generate a plain text file of new files in the last X days, that would be even better. This has been tested only on the Kobo Glo with firmware 2.1.5.

Here's how it works:

Add to .kobo/koboplugins.ini:
Code:
CSync=Plugin::connectWifi()>>Plugin::executeSystemCmd("/mnt/onboard/run.sh &")
Note that this seems to force this entry in the menu to appear as "WiFi" - it looks like there's a bug in the plugin that forces the label to be "WiFi" whenever it's used, in a command sequence like this, or on its own. If you already have a WiFi entry in your menu, remember they appear alphabetically, so make sure you select the correct one.

Now create the following scripts in the Kobo root folder (i.e. the root folder when you access it as a USB drive):

run.sh
Code:
#!/bin/sh
touch /mnt/onboard/started_sync
nohup /mnt/onboard/sync.sh &
(I use this intermediate file to call the actual sync script, because if I try to call it directly, even with Plugin::executeSystemCmd("nohup sync.sh &"), the menu seems to freeze until the whole thing finishes.)


sync.sh
Code:
#!/bin/sh

# Add your HTTP server details here
LIST_URL="http://192.168.0.123/books/filelist.php";
PING_IP="192.168.0.123"

ATTEMPTS=30

#Uncomment the next line to always download files in the list (i.e. ignore 
existing downloaded file, save with unique name)
# NOTE: for testing only! Will create duplicate books on your device
#ALWAYS_DL=1

# wait for wifi network to come online; give up after $ATTEMPTS tries
tries=1
while ! ping -c 1 -q $PING_IP; do 
   if [ "$tries" -gt "$ATTEMPTS" ] ; then 
     timestamp=`date`
     echo "sync: aborted after $ATTEMPTS connection attempts ($timestamp)" >>/mnt/onboard/sync.log
     break; 
   fi
   sleep 2
   tries=$(( $tries + 1 ))
done

   
successes=0
LIST=`/usr/bin/wget -q -O - $LIST_URL`
IFS=$'\n'  # make the for loop break up the list on newlines

for url in $LIST; do
  filename=`echo $url|sed -e 's|.*/||g'`
  if [ $ALWAYS_DL ] ; then
    timestamp=`date +%Y%m%d-%k%M%S`
    if /usr/bin/wget "$url" -O "/mnt/onboard/$timestamp-$filename" >>/mnt/onboard/wget.log 2>&1 ; then
      successes=$(( $successes + 1 ))                                                                                                                 
      echo "Downloaded (forced) $url to $timestamp-$filename" >>/mnt/onboard/sync.log
    else
      timestamp=`date`                                                                                                                                
      echo "Failed to force download $url ($timestamp)" >>/mnt/onboard/sync.log                                                                
    fi
  else
    if /usr/bin/wget "$url" -O "/mnt/onboard/$filename"  >>/mnt/onboard/wget.log 2>&1 ; then 
      successes=$(( $successes + 1 ))
      echo "Downloaded $url" >>/mnt/onboard/sync.log
    else
      timestamp=`date`
      echo "Failed to download $url (file exists? $timestamp)" >>/mnt/onboard/sync.log
    fi
  fi
done

if [ "$successes" -gt "0" ] ; then 
  timestamp=`date`
  echo "Downloaded $successes files ($timestamp)" >>/mnt/onboard/sync.log
  nohup /mnt/onboard/reindex.sh &
fi
(You will need to put the correct address for your HTTP server at the appropriate place at the top of the script. Note that this script first checks that it can reach the server on the network. This is because the command sequence we defined in the menu turns on the Wifi but does not wait for the connection to be established before running the next command, i.e. this sync.sh script. So the script itself must keep checking and waiting until the network is up. It will give up after 30 attempts, about 60 seconds.

reindex.sh
Code:
#!/bin/sh
echo usb plug add >> /tmp/nickel-hardware-status 
sleep 10 
echo usb plug remove >> /tmp/nickel-hardware-status
This forces the Kobo to add any files downloaded to the Library by faking a USB connection and disconnection (found here: http://a3nm.net/blog/fnacbook_kobo_more_hacking.html). It works but requires you to tap 'connect' on the dialog it brings up on the screen, and then automatically disconnects and indexes a few seconds later. It works, but if anyone knows how to trigger a reindex in a less clunky and interactive way, please let me know.

These files all need to be set as executable, i.e. "chmod 755".

Here is the PHP script which will create the expected list of file URLs to download; the epub files must in the same directory as the script.
Code:
<?php
$path = substr($_SERVER[PHP_SELF], 0, strrpos($_SERVER[PHP_SELF], '/'));
$find = 'find . -iname \\*.epub'; //-mtime 0 ';

$ar = array();
exec($find, $ar);

foreach($ar as $file){
 $url = "http://" . $_SERVER['SERVER_NAME']  . $path . '/' . str_replace(' ', '%20', substr($file, 2));
 print "$url\n";
 //print "<a href=\"$url\">$url</a><br>";

}
?>
When you test this PHP script in your browser you should get a list of Epub file URLS, separated by newlines (they will all appear to be on the same line in your browser). If you don't see them, make sure you have some files in the same directory as the script, and check that they're accessible - uncomment the second "print" statement at the end to test that the links work.

Whew. Ok, once all that is in place, you should be able to automatically download files to your Kobo and have them automatically indexed (after the fake USB-connect sequence). The script will not re-download files that it has already saved to the Kobo unless you uncomment the ALWAYS_DL line in sync.sh, but note that this creates unique filenames each time it downloads a file, so you'll get multiple versions of the same book in your library.

The script does some logging to wget.log and sync.log in the same root directory as the scripts themselves.

TODO:

- Less interactive way to force checking/reindexing of new files. Anyone know if this can be done via a shell command?

- Neater way to make sure wifi is available before doing download

- Fix label bug that causes menu item to appear as "WiFi"

- Ideally it would be good to either serve the files from the Calibre server, if it can be made to generate a simple enough list of URLs of new files. Alternately, it would be good to be able to remotely host the files so they can be synced even when your own computer is off, i.e. Dropbox or similar; however with Dropbox there's no way to generate the list of file URLs, and no way to list the contents of a web folder to get all the files. If anyone can think of a generic alternative that would work in this case, that would be good.

- Text updates at the top of screen - currently it gets stuck on "Connecting", I think this is the Wifi command. Not sure how to change the this text.

- A way to extract emailed articles and save them as epub files, so they can be downloaded using this process, i.e. make use of browser extensions that allow you to "email this page to your kindle"... any ideas?


Attached is a zip file with the above scripts that need to be extracted into the root folder, and the PHP script for your server. The Kobo shell scripts will probably still need to have the correct execute permissions set - this will be hard if you haven't enabled telnet for the device.

Let me know how this works if you try it out.
Attached Files
File Type: zip kobo-sync-scripts.zip (1.7 KB, 59 views)

Last edited by textchimp; 12-04-2012 at 09:30 PM.
textchimp is offline   Reply With Quote
Old 12-04-2012, 03:37 PM   #217
vlad59
Addict
vlad59 can name that ebook in five wordsvlad59 can name that ebook in five wordsvlad59 can name that ebook in five wordsvlad59 can name that ebook in five wordsvlad59 can name that ebook in five wordsvlad59 can name that ebook in five wordsvlad59 can name that ebook in five wordsvlad59 can name that ebook in five wordsvlad59 can name that ebook in five wordsvlad59 can name that ebook in five wordsvlad59 can name that ebook in five words
 
Posts: 353
Karma: 37761
Join Date: Sep 2011
Device: Kobo eReader Touch, Kobo Aura HD
Great work !

If you want I can add a special page in Cops (http://blog.slucas.fr/en/oss/calibre-opds-php-server) that produce the correct file format.

It's been a while since I've used the plugin, but you had a goo idea.
vlad59 is offline   Reply With Quote
Old 12-05-2012, 03:06 PM   #218
benjaminries
Junior Member
benjaminries began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Dec 2012
Device: Kobo Glo
Got a Kobo Glo last weekend. Running firmware 2.1.5. People on this thread seem to confirm that the plugin interface works on the Glo with this firmware.

I followed the instructions - connected my Glo via USB, unzipped both files to the .kobo directory, and ejected the device from my PC. I think the .tgz file is being read, because when I do this, the screen says that the software is updating.

My issue is that after that, the old stock interface is still there, with no changes. I open a book, and close it. Still no change. I shut down the Kobo, boot it back up, open and close a book, and still nothing.

I've tried this with my own *.ini content, and with an unmodified *.ini file downloaded from the first post of this thread. Neither has worked.

What am I doing wrong?
benjaminries is offline   Reply With Quote
Old 12-06-2012, 10:10 AM   #219
textchimp
Member
textchimp has a complete set of Star Wars action figures.textchimp has a complete set of Star Wars action figures.textchimp has a complete set of Star Wars action figures.
 
Posts: 15
Karma: 250
Join Date: Nov 2012
Device: Kobo Glo
Does anyone know if there's a command available on the embedded Linux running on the Kobo Touch/Glo which could send a file to a remote host? Ideally something like an automated FTP file send (i.e. ncftpput - but clearly that's not available), although even making an HTTP POST would probably be enough.

I'm trying to write a script to send all the annotations/highlights on the device to a remote server.
textchimp is offline   Reply With Quote
Old 12-06-2012, 10:27 AM   #220
SeigneurAo
Connoisseur
SeigneurAo knows the complete value of PI to the endSeigneurAo knows the complete value of PI to the endSeigneurAo knows the complete value of PI to the endSeigneurAo knows the complete value of PI to the endSeigneurAo knows the complete value of PI to the endSeigneurAo knows the complete value of PI to the endSeigneurAo knows the complete value of PI to the endSeigneurAo knows the complete value of PI to the endSeigneurAo knows the complete value of PI to the endSeigneurAo knows the complete value of PI to the endSeigneurAo knows the complete value of PI to the end
 
SeigneurAo's Avatar
 
Posts: 62
Karma: 31414
Join Date: Sep 2012
Location: France
Device: Kobo Glo
Quote:
Originally Posted by textchimp View Post
Does anyone know if there's a command available on the embedded Linux running on the Kobo Touch/Glo which could send a file to a remote host? Ideally something like an automated FTP file send (i.e. ncftpput - but clearly that's not available), although even making an HTTP POST would probably be enough.

I'm trying to write a script to send all the annotations/highlights on the device to a remote server.
Maybe wget ?
Quite widespread and basic command on Linux systems.
SeigneurAo is offline   Reply With Quote
Old 12-06-2012, 03:30 PM   #221
koenieee
Enthusiast
koenieee doesn't litterkoenieee doesn't litterkoenieee doesn't litter
 
Posts: 49
Karma: 208
Join Date: Nov 2012
Device: Kobo Glo
Do we have curl access to our Kobo's?
Then we could use this: http://superuser.com/questions/86043...part-form-data (Second post)

I was thinking to create some sort script that synces epubs booksmarks (not the offical kobo books) with my android smartphone.

Does anyone know where the Kobo saves the bookmarks?
In the sqllite database I guess?
koenieee is offline   Reply With Quote
Old 12-06-2012, 05:24 PM   #222
textchimp
Member
textchimp has a complete set of Star Wars action figures.textchimp has a complete set of Star Wars action figures.textchimp has a complete set of Star Wars action figures.
 
Posts: 15
Karma: 250
Join Date: Nov 2012
Device: Kobo Glo
Sending highlights/annotations to remote server

Ha, ok yes, wget again. I've got this to work, again as a proof-of-concept, but the version of wget on the Kobo is limited - it does not have the --post-file=FILE command line option, but it does have (undocumented) the --post-data="DATA" option. This means you can send some post data, but only via the command line, and it seems to limit how much data you can send. (Anyone got a better idea?)

Anyway, here's a working example to grab all the annotation data from /mnt/onboard/Digital\ Editions/Annotations and post it to a PHP script on a server.

First create a menu entry in koboplugins.php like:

Code:
CSync=Plugin::connectWifi()>>Plugin::executeSystemCmd("/mnt/onboard/run-notesync.sh &")
Then as with the sync script above, we use an intermediate script to prevent the menu freezing up:

run-notesync.sh:
Code:
#!/bin/sh
touch /mnt/onboard/started_notesync
nohup /mnt/onboard/notesync.sh &
notesync.sh:
Code:
#!/bin/sh

ATTEMPTS=40

# change the addresses below for your server
REMOTE_URL="http://serveraddress/path-to/post.php"
PING_ADDRESS="serveraddress"
POSTDATA=""

# wait for wifi network to come online; give up after $ATTEMPTS tries
tries=1
while ! ping -c 1 -q $PING_ADDRESS; do
   if [ "$tries" -gt "$ATTEMPTS" ] ; then
      timestamp=`date`
      echo "sync: aborted after $ATTEMPTS connection attempts ($timestamp)" >>/mnt/onboard/notesync.log
      exit; 
   fi
   sleep 2
   tries=$(( $tries + 1 ))
done


POSTDATA=""
FILES=`find "/mnt/onboard/Digital Editions/Annotations/" -iname \*.annot -exec grep -l '<text>' {} \;`
IFS=$'\n'
for file in $FILES; do
  POSTDATA=${POSTDATA}`awk '/<dc:title>/ ||  /<dc:creator>/ || /<text>/ || /<\/text>/' "$file"`
  POSTDATA="${POSTDATA}\n\n"
done

#echo -e "$POSTDATA"

wget $REMOTE_URL -O /mnt/onboard/notesync.log --post-data="content=$POSTDATA"
timestamp=`date`
echo -e "\n\nSent on $timestamp \n" >>/mnt/onboard/notesync.log
This will extract just the title, creator and text tags from all the annotation data, in the interest of saving upload size via the limited wget command line option. As with the download script in my sync script post, it will first test the network connection until it successfully pings the remote host, otherwise it will quit.

(NOTE: it looks like the way the notes are saved in these .annot files ignores paragraph breaks - it just runs text from different paragraphs together. Annoying! In any case, the awk filtering done in the script will ignore lines without a <text> or </text> tag; needs a better multiline regex...)

On the server end, you need a simple PHP script like this:
post.php
Code:
<?php
if(isset($_POST['content'])) {
  file_put_contents('notes.html', $_POST['content']);
  exit;
}
?>
Then you can see your notes at http://server-address/path-to/notes.html.

Last edited by textchimp; 12-06-2012 at 10:25 PM.
textchimp is offline   Reply With Quote
Old 12-06-2012, 06:41 PM   #223
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 6,090
Karma: 6238033
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo Touch, Kobo Glo
Quote:
Originally Posted by koenieee View Post
I was thinking to create some sort script that synces epubs booksmarks (not the offical kobo books) with my android smartphone.

Does anyone know where the Kobo saves the bookmarks?
In the sqllite database I guess?
The bookmarks are in three places depending on what you want.

The annotations and highlights are in the annot file that textchimp is using. These are also in a database table called "Bookmark". Hopefully any ADE based reader could reuse the file. Also, the file on the Kobo's takes precedence. If you delete the records in the table, the annotations are loaded from the file. If you delete the file, the annotations are removed.

The current place in the book is stored in the "content" table. Each book has a row with ContentType=6. Books that are being read have a value in the column "ChapterIDBookmarked". This has a reference to the current chapter. The reference is the key to the content table for a ContentType=9 row. This row has a value in the column "adobe_location". This is the current location in the chapter. The following SQL will show the current position for all sideloaded books:

Code:
SELECT c1.Title, c1.Attribution, c1.DateLastRead, c1.ChapterIDBookmarked, c1.ReadStatus, c1.___PercentRead, c2.ContentId, c2.Title, c2.adobe_location
FROM content c1 join content c2 on c1.ChapterIDBookmarked = c2.ContentID 
where c2.bookid like 'file:///mnt/onboard%'
Decoding the value in adobe_location is the only problem in this data. An example for me is "OEBPS/138508.xhtml#point(/1/4/278/1:222)". The first part is the name of a file inside the epub. The point must translate to some position in the file, but I haven't bothered to work out exactly what it means.
davidfor is offline   Reply With Quote
Old 12-06-2012, 08:07 PM   #224
sensoria
Junior Member
sensoria began at the beginning.
 
Posts: 7
Karma: 10
Join Date: Dec 2012
Location: Canada
Device: Kobo Touch
Just wanted to say many, many thanks for this plugin. It's making my Kobo experience so much better. :-) Love the enhanced reading footer especially.
sensoria is offline   Reply With Quote
Old 12-07-2012, 03:57 AM   #225
jkgeyti
Member
jkgeyti is a marvel to beholdjkgeyti is a marvel to beholdjkgeyti is a marvel to beholdjkgeyti is a marvel to beholdjkgeyti is a marvel to beholdjkgeyti is a marvel to beholdjkgeyti is a marvel to beholdjkgeyti is a marvel to beholdjkgeyti is a marvel to beholdjkgeyti is a marvel to beholdjkgeyti is a marvel to behold
 
Posts: 17
Karma: 11628
Join Date: Oct 2012
Device: Kobo Touch
textchimp: I'm also using the same approach to "syncing" books. You may want to use the plugin's command chaining and its sync mechanism instead of simulating a usb connecting. Also, the busybox wget will not work with secure connections. So you may want to use curl instead.

I'm currently also using a simple php script with a simple interface to add URLs that I want the kobo to download. When I get some free time, I hope to do Dropbox syncing using their REST api.
jkgeyti is offline   Reply With Quote
Reply

Tags
kobo, plugin

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Where can i get firmware 2.2.1 alpha for pocketbook 912 johannes866 PocketBook 1 02-13-2012 05:57 AM
Firmware 2.0.3 Pocketbook IQ - another firmware is release for IQ tvpupsik PocketBook 26 12-13-2010 04:03 PM
DR1000 Chinese Vertical Layout uds plugin plaintext Release spuggy iRex 4 09-18-2010 03:00 PM
dotReader alpha release next month with future iLiad support possible Alexander Turcic News 7 09-28-2006 01:24 PM
OpenBerg Milestone Release 1 alpha released Colin Dunstan Reading and Management 0 07-13-2005 08:41 AM


All times are GMT -4. The time now is 08:06 PM.


MobileRead.com is a privately owned, operated and funded community.