Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Readers > Amazon Kindle > Kindle Developer's Corner

Notices

Reply
 
Thread Tools Search this Thread
Old 04-05-2012, 01:54 PM   #1
pwr
Member
pwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with others
 
pwr's Avatar
 
Posts: 18
Karma: 2602
Join Date: Mar 2012
Device: Kindle 4NT
Wink Kindle Store proxy for Kindle4NT

Hello,

I've been working on a Kindle Store proxy for the Kindle 4 (NonTouch), which works with Calibre. If set-up correctly (which may take some doing), it integrates (as seamlessly as it can) the list of books present in the Calibre library with the list of books the Amazon returns, allowing you to download books, from the Calibre library to the Kindle, as if they were on your Amazon account.

It is not feature-complete -- right now last-position-read, notes and highlights are not saved. I'll be implementing them as soon as I figure out the MBP format .

The source code (KSP is written in Python 3) is available on GitHub. The installation procedure and the changes that need to be made on the device are documented in detail; there are also some scripts to help with changing the device configuration.

I'm not sure how useful it may be to other people -- I've found it quite useful . Right now it only works with the Kindle4NT because it's the only device I have to test it with; I'm guessing it could be easily deployed on a Kindle 3 (Keyboard) as well. Not so sure about Kindle Touch, as I understand it's quite different from the K4NT.

If you have any feedback, ideas or suggestions for further development, I'd love to hear them .

Cheers,
-pwr
pwr is offline   Reply With Quote
Old 04-08-2012, 06:23 AM   #2
TRManderson
Junior Member
TRManderson began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Apr 2012
Device: Kindle 4 NT
[redacted]

Last edited by TRManderson; 05-29-2012 at 12:00 AM. Reason: delete
TRManderson is offline   Reply With Quote
Old 04-27-2012, 07:56 AM   #3
pwr
Member
pwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with others
 
pwr's Avatar
 
Posts: 18
Karma: 2602
Join Date: Mar 2012
Device: Kindle 4NT
Talking KSP updated, added support for last-read and annotations

Hello,

I've updated my Kindle Store Proxy (https://github.com/pwr/KSP). Last-read position and annotations are saved in KSP's database, so now deleting a Calibre book and re-downloading it will keep the last-read and any notes or bookrmarks you've made on it.

I've also tried to simplify the device registration process, though it still is somewhat involved.

Based on feedback I've received so far, I'm focusing on supporting more devices, in particular Kindle for Android, especially for the last-read position feature.

Cheers,
-pwr
pwr is offline   Reply With Quote
Old 04-29-2012, 03:31 PM   #4
qlob
Official Lurker
qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.
 
qlob's Avatar
 
Posts: 1,050
Karma: 7096675
Join Date: Apr 2012
Device: Kindle 3.4
***raises hand to volunteer a k3 for testing***
qlob is offline   Reply With Quote
Old 04-30-2012, 03:04 AM   #5
pwr
Member
pwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with others
 
pwr's Avatar
 
Posts: 18
Karma: 2602
Join Date: Mar 2012
Device: Kindle 4NT
Smile

Quote:
Originally Posted by qlob View Post
***raises hand to volunteer a k3 for testing***
Shiny.

Working with the Kindle 4 (NT), there are a few files and components that are used on the device to make KSP work (I'm assuming you have jailbroken the Kindle 3 and can SSH in):
  • /proc/usid - should contain the device serial (in the form B00...)
  • /var/local/java/prefs/com.amazon.ebook.framework/ServerConfig - a configuration file, may contain the urls the device uses to talk to Amazon's services; this is one of the files that need changed
  • /opt/amazon/ebook/config/ServerConfig.conf - the master configuration for service urls; does not need to be modified, but helps as a reference to what the urls look like; it has entries like url.todo=https://todo-g7g.amazon.com/...
  • /var/local/java/prefs/certs/client.p12 - a java keystore containing the device's client SSL certificate; needs to be copied in your KSP's deployment, in the ./db folder, and named device_serial.p12
  • /usr/java/lib/security/cacerts - CA certificates used by the Java GUI
  • /etc/ssl/certs/ca-certificates.crt - CA certificates used by the book downloading service

To work with the Kindle 3, these files (or some equivalent of them) would be necessary on the device. I think -- I'm not at all sure the Kindle 3 works the same as the Kindle 4. Could you check they exist, and what their contents look like? If /opt/amazon/ebook/config/ServerConfig.conf exists, could you PM me its contents?

The manual configuration procedure (for Kindle 4 NT) is detailed at https://github.com/pwr/KSP/blob/master/docs/devices.md, in the "Configuration changes" section (btw, I recomend reading all the docs there before fiddling with your Kindle). There are a couple of scripts I've written to do the configuration automatically on the Kindle 4, but don't run them on the Kindle 3 -- quite likely they will not work (or worse).

When locally testing the configuration, you can probably just use a HTTP server_url, and ignore the parts about HTTPS server ceriticates and CA files -- but would still love to know if those files exist on the device. However, if you do make it work though, and plan to use your device with KSP all the time, I strongly advice using a HTTPS url for KSP -- the traffic between the device and KSP can be quite sensitive.

Then, assuming you've convinced your Kindle to talk to the KSP daemon, I would love to see some of KSPs logs (in ./logs/), to make sure it's understanding and handling the calls it receives properly. If not, seeing the logs will help me do the appropiate fixes.

If you have any questions, or find the documentation unhelpful, please let me know. I've tried to write the documentation as complete and clear as I could, but it's quite likely I've failed in both .

Many thanks

Cheers,
-pwr
pwr is offline   Reply With Quote
Old 04-30-2012, 10:09 AM   #6
akleiw
Member
akleiw began at the beginning.
 
Posts: 19
Karma: 12
Join Date: Sep 2011
Device: Kindle 4.1.0
pwr,
this looks really useful, and your documentation is great. Could you screenshot what it looks like on the Kindle?
akleiw is offline   Reply With Quote
Old 04-30-2012, 03:05 PM   #7
pwr
Member
pwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with others
 
pwr's Avatar
 
Posts: 18
Karma: 2602
Join Date: Mar 2012
Device: Kindle 4NT
Erm, it does not look in any way different. That's kind of the point -- to make the Kindle see the books in the Calibre library as if there were on your Amazon account. The device functions exactly the same -- I think I've covered pretty much all the functions, save the annotations sharing. It uses the same http calls to talk upstream, it's just that it talks to KSP instead of directly to Amazon.

Cheers,
-pwr
pwr is offline   Reply With Quote
Old 05-04-2012, 11:30 PM   #8
qlob
Official Lurker
qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.
 
qlob's Avatar
 
Posts: 1,050
Karma: 7096675
Join Date: Apr 2012
Device: Kindle 3.4
Sorry for the wait, i recently bricked my kindle... Anyways,
/proc/usid exists,
/var/local/java/prefs/com.amazon.ebook.framework/ServerConfig doesnt,
/opt/amazon/ebook/config/ServerConfig.conf does,
/var/local/java/prefs/certs/client.p12 does,
/usr/java/lib/security/cacerts does,
/etc/ssl/certs/ca-certificates.crt does. I will pm you the contents of serverconfig.conf asaic.
qlob is offline   Reply With Quote
Old 05-05-2012, 01:01 AM   #9
geekmaster
Carpe diem, c'est la vie.
geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.geekmaster ought to be getting tired of karma fortunes by now.
 
geekmaster's Avatar
 
Posts: 6,433
Karma: 10773668
Join Date: Nov 2011
Location: Multiverse 6627A
Device: K1 to PW3
Quote:
Originally Posted by qlob View Post
Sorry for the wait, i recently bricked my kindle...
I lost count of the times one or more of my kindles sat bricked until I could find a NEW way to debrick it, which is part of the driving force for my various improving solutions to debricking posted in this forum. Unfortunately, people keep finding new ways to brick them that resist known debricking methods.

While the "success report" list keeps growing, there is not yet a "beyond hopeless" list.


Last edited by geekmaster; 05-05-2012 at 01:10 AM.
geekmaster is offline   Reply With Quote
Old 05-05-2012, 03:33 PM   #10
qlob
Official Lurker
qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.qlob ought to be getting tired of karma fortunes by now.
 
qlob's Avatar
 
Posts: 1,050
Karma: 7096675
Join Date: Apr 2012
Device: Kindle 3.4
Quote:
Originally Posted by geekmaster View Post
I lost count of the times one or more of my kindles sat bricked until I could find a NEW way to debrick it, which is part of the driving force for my various improving solutions to debricking posted in this forum. Unfortunately, people keep finding new ways to brick them that resist known debricking methods.

While the "success report" list keeps growing, there is not yet a "beyond hopeless" list.

Yup. If it werent for this forum, i would probably have lost interest in the kindle...
However, my kindle still is not booting to the framework, i am just currently in diags with myts....
qlob is offline   Reply With Quote
Old 05-06-2012, 05:08 AM   #11
pwr
Member
pwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with others
 
pwr's Avatar
 
Posts: 18
Karma: 2602
Join Date: Mar 2012
Device: Kindle 4NT
Quote:
Originally Posted by qlob View Post
/proc/usid exists,
/var/local/java/prefs/com.amazon.ebook.framework/ServerConfig doesnt,
/opt/amazon/ebook/config/ServerConfig.conf does,
/var/local/java/prefs/certs/client.p12 does,
/usr/java/lib/security/cacerts does,
/etc/ssl/certs/ca-certificates.crt does
Hey,

Thanks for the info.

ServerConfig missing in /var/local is a bit funny, at least on my Kindle 4 there's always something there, even before messing with it. But I've done a bit of digging on the 'net and I think it's no problem, the Kindle 3 framework still uses the file if it exists. So I've updated the main deployment script (RUNME.sh) to work even if the file is not found.

Other than that, things look to be working close enough to the Kindle 4 from what I can tell (I've managed to have a look into the java implementation as well). I've already found a default ServerConfig.conf for Kindle 3, and it looks almost the same as the one in Kindle 4.

So I guess you could give it a try installing the KSP config on your Kindle 3. The main script is RUNME.sh in tools/kindle4nt/ksp-config/. You will have to copy it to the device (preferably to /mnt/us/), modify it a bit (the SERVER_URL= line), and run it from a SSH console:

Code:
./RUNME.sh 2>&1 | tee RUNME.log
If you've configured KSP to run over HTTPS, it will look for and launch the get_certificates.sh and keytool.sh scripts, so you need them on the device as well.

If you want to be extra careful about it (and have a lot of patience), you could probably copy and run each line from the scripts, one by one, and check the results after each one. Or you could just add a second line to each script, "set -v", which will log each and every line before running it -- the resulting log file would be very informative.

Thank you very much for your help, I appreciate it especially after you've already bricked your Kindle once .

Cheers,
-pwr
pwr is offline   Reply With Quote
Old 05-06-2012, 02:19 PM   #12
akleiw
Member
akleiw began at the beginning.
 
Posts: 19
Karma: 12
Join Date: Sep 2011
Device: Kindle 4.1.0
Hi,
Unfortunately I couldn't get it to work properly on my machine. I'll PM you my config files and server.log (the other two logs are empty). You can see the errors appearing in the log once the Kindle is turned on at 21:16.

Here's my set-up:
Kindle 4 fw. 4.0.1
Windows 7 32bit
Python 3.2.2
Client certificate present in the ./db
akleiw is offline   Reply With Quote
Old 05-06-2012, 07:29 PM   #13
pwr
Member
pwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with others
 
pwr's Avatar
 
Posts: 18
Karma: 2602
Join Date: Mar 2012
Device: Kindle 4NT
Hm, my bad. There was an error in the code for registering new devices. Sorry about that, did some changes there to support Kindle for PC and forgot to test registration with my Kindle as well.

I've fixed the code and tested properly this time on my Kindle. You should pull the latest version from git before trying again.

Other than that, the configuration looks fine in config.py and ServerConfig as well.

Thanks for testing it .

Cheers,
-pwr
pwr is offline   Reply With Quote
Old 05-07-2012, 06:02 PM   #14
akleiw
Member
akleiw began at the beginning.
 
Posts: 19
Karma: 12
Join Date: Sep 2011
Device: Kindle 4.1.0
Thank you, it's working!

Well, it's not perfect yet, because I can't download anything from Amazon's cloud (it's stuck as pending) but I've successfully loaded some books from my calibre library. I'll report back when I take a better look at this issue.

I've sent you a pull request on Github with a tiny patch I had to apply in order to make KSP work with my Kindle. It 's my first time using Git, sorry if I've mixed something up.
akleiw is offline   Reply With Quote
Old 05-07-2012, 06:52 PM   #15
pwr
Member
pwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with otherspwr plays well with others
 
pwr's Avatar
 
Posts: 18
Karma: 2602
Join Date: Mar 2012
Device: Kindle 4NT
Thank you for the pull request, that part of code isn't reached in my deployment and obviously haven't tested it properly (I use Linux with command-line openssl). I've merged the request into my tree.

Quote:
Originally Posted by akleiw View Post
Well, it's not perfect yet, because I can't download anything from Amazon's cloud (it's stuck as pending) but I've successfully loaded some books from my calibre library. I'll report back when I take a better look at this issue.
Hm, Amazon books should be downloaded directly by the device -- KSP should be replying with a 302 redirect pointing to the Amazon URL if the device tries to download a book not in Calibre's library. It works fine for me both on the Kindle 4 and in Kindle for PC.

If the download gets stuck somehow (happened to me a lot during development, especially when KSP was throwing some exception), you can try restarting the device and re-downloading the book. If it gets stuck again, then it is indeeed a problem. Is it stuck at 0% or 100%?

Cheers,
-pwr
pwr is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
eBooks in Different Languages Kindle4nt kobestarr Amazon Kindle 2 02-06-2012 12:24 PM
make kindle firmware use a proxy server Nigovic Kindle Developer's Corner 3 10-19-2011 07:49 AM
Using calibre in a proxy. fetching metadata proxy authentication required shiftymorgan Calibre 8 05-13-2011 03:45 AM
K4 Mac or PC Using Kindle for PC with Proxy evilK Amazon Kindle 21 07-28-2010 07:23 AM
Proxy authentication problems (I have no proxy) fogus Calibre 16 03-08-2009 01:27 PM


All times are GMT -4. The time now is 07:36 AM.


MobileRead.com is a privately owned, operated and funded community.