Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 06-01-2009, 11:13 PM   #1
ldolse
Wizard
ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.
 
Posts: 1,337
Karma: 123455
Join Date: Apr 2009
Location: Malaysia
Device: PRS-650, iPhone
Feature Request: ipod touch/iphone kindle.app support

Note I didn't title this thread quite right, it should read:
Feature Request: Jailbroken ipod touch/iphone kindle.app support

Hi,

Ever since Amazon allowed iphones/ipod touch users to purchase ebooks I've been using them as my primary source of purchased ebooks. While I really dislike the draconian policies they have with drm and delivery of paid content, I've used them in general for years and their prices are good.

The real problem is that I read these on my Sony, which means I've got to jump through quite a few hoops to get the content there:
  1. Jailbreak the ipod/iphone (one time event)
  2. Acquire software which allows me to browse the device filessytem (one time event)
  3. Launch the above software
  4. find the buried location of kindle.app and it's ebook library
  5. Copy the code named prc files to my local disk
  6. Go through each file one by one until I figure out which one is the latest book I've purchased
  7. Get the serial number of the ipod touch and use it to get the mobipocket ID (one time event)
  8. Strip the DRM
  9. Load the book into Calibre
  10. Send it to the sony

Needless to say that's a bit complicated, and I'd love it if Calibre could uncomplicate it for me along the lines of something like this:
  1. Plug the jailbroken ipod/iphone in, Calibre recognizes it as a device
  2. Calibre allows me to browse whatever books are on the device and transfers them to my library

At that point they would still be DRM'd, but the fact that I could stick them into the library from the device intuitively would make all the difference in the world. Stripping the DRM and replacing the file in the calibre library is relatively trivial.

Of course it would be major bonus points if Calibre could strip the DRM in the process - but I understand the reasons behind not supporting it in the official release or explicitly on these forums. One could always link a plugin modification over on darkreverser's blog or some other location not affiliated with mobileread.


I'm thinking this FR should apply to quite a few people but not entirely certain, so looking for comments.

I also tried looking into downloading the files directly from amazon's site by capturing the ipod's traffic on my network, but amazon uses HTTPS with client certificate auth to download the books. I looked all through the kindle.app directory, but I couldn't find any keypair being used as a client cert, I can only assume that Amazon is embedding the keypair in the actual binary. If I could get the client cert I could actually decrypt the SSL traffic pretty easily with the tools I have to determine what the app is doing.

Last edited by ldolse; 06-02-2009 at 12:43 AM.
ldolse is offline   Reply With Quote
Old 06-01-2009, 11:47 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,991
Karma: 5036765
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I don't have an iPhone, so I for one can't do this, but patches are welcome Incidentally calibre's plugin frameworks makes removing MOBI DRM automatic
kovidgoyal is offline   Reply With Quote
Old 06-02-2009, 12:00 AM   #3
Seabound
Steerage Class
Seabound ought to be getting tired of karma fortunes by now.Seabound ought to be getting tired of karma fortunes by now.Seabound ought to be getting tired of karma fortunes by now.Seabound ought to be getting tired of karma fortunes by now.Seabound ought to be getting tired of karma fortunes by now.Seabound ought to be getting tired of karma fortunes by now.Seabound ought to be getting tired of karma fortunes by now.Seabound ought to be getting tired of karma fortunes by now.Seabound ought to be getting tired of karma fortunes by now.Seabound ought to be getting tired of karma fortunes by now.Seabound ought to be getting tired of karma fortunes by now.
 
Seabound's Avatar
 
Posts: 711
Karma: 505995
Join Date: Mar 2008
Location: Pacific Northwest, USA
Device: Won't fit here anymore, see sig for a list of liseuses.
Quote:
Originally Posted by kovidgoyal View Post
Incidentally calibre's plugin frameworks makes removing MOBI DRM automatic
Wow, I've been missing some news. When did that automated Mobi de-DRM feature happen?

BTW, I noticed the Ph.D.--congratulations Dr. Goyal!
Seabound is offline   Reply With Quote
Old 06-02-2009, 12:13 AM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,991
Karma: 5036765
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by Seabound View Post
Wow, I've been missing some news. When did that automated Mobi de-DRM feature happen?

BTW, I noticed the Ph.D.--congratulations Dr. Goyal!
Thanks

MOBI DRM removal is automatic in calibre if you can find a DRM removing plugin for calibre

Last edited by kovidgoyal; 06-02-2009 at 12:23 AM.
kovidgoyal is offline   Reply With Quote
Old 06-02-2009, 03:15 AM   #5
ldolse
Wizard
ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.
 
Posts: 1,337
Karma: 123455
Join Date: Apr 2009
Location: Malaysia
Device: PRS-650, iPhone
Well, I looked through the files a bit more in the app, I think I can probably take a stab at a start, but not sure how far I'll get. The full books are all listed in the attached xml file. Downloaded samples don't appear in the xml or any other support files for some reason, possibly because they don't have DRM. The filename of the book is the ASIN id plus an extra static extension. So It should be relatively straightforward to parse the xml to show the current books on the iphone.

The big issue is that the iphone doesn't mount as a proper usb disk/filesystem. There are a few filebrowsers that let you access the system using some special tricks, but I don't see a cross-platform library doing this.

I think for now I'm going to try out an approach where I share the filesystem over wifi using Netatalk. It allows you to create an AFP share from the iphone.

If it helps I could post the whole kindle.app directory sans paid books.
Attached Files
File Type: xml KindleSyncMetadataCache.xml (7.1 KB, 171 views)
ldolse is offline   Reply With Quote
Old 06-02-2009, 06:26 AM   #6
ldolse
Wizard
ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.
 
Posts: 1,337
Karma: 123455
Join Date: Apr 2009
Location: Malaysia
Device: PRS-650, iPhone
Some more info:

After installing NetaTalk the ipod touch is automatically discovered on the network as an AFP share, with two volumes, one called root, one called home. Once root is mounted the path on OS X is the following:

Code:
/Volumes/root/var/mobile/Applications/939E5CA6-B556-4C0E-800C-1A7#A2D
I'm assuming that 939E5CA6-B556-4C0E-800C-1A7#A2D is the universal folder name for the kindle app across all iphones/ipod touches, but I would need some other users who have jailbroken their iphones to corroborate that. If it's not each subfolder in /Applications/ has a name that looks like that, the subfolders would need to be searched for one named Kindle.app to confirm the correct folder for that device.

The xml file I attached previously is located within the app folder at:
Code:
Relative Path from program directory:
/Documents/cache/KindleSyncMetadataCache.xml

Full Path (OS X):
/Volumes/root/var/mobile/Applications/939E5CA6-B556-4C0E-800C-1A7#A2D/Documents/cache/KindleSyncMetadataCache.xml
If you look at single entry from the XML:
Code:
<meta_data><ASIN>B000FBJCJE</ASIN><title>Snow Crash</title><authors><author>Stephenson, Neal</author></authors><publishers><publisher>Spectra</publisher></publishers><publication_date>2003-08-26T00:00:00+0000</publication_date><cde_contenttype>EBOK</cde_contenttype><content_type></content_type></meta_data>
You'll see the ASIN is B000FBJCJE. The book filename can be constructed from this by appending _EBOK.prc:
Code:
Filename:
B000FBJCJE_EBOK.prc

Relative Path:
/Documents/eBooks/B000FBJCJE_EBOK.prc

Full Path (OS X):
/Volumes/root/var/mobile/Applications/939E5CA6-B556-4C0E-800C-1A7#A2D/Documents/eBooks/B000FBJCJE_EBOK.prc
All the ebooks are in that same folder. I suppose another option would be to just parse the metadata of all the files, but that seems like a lot more work.


The biggest problem with the AFP approach is that it's not natively available on Windows or Linux, but I think that it can be installed. The alternative would be to use OpenSSh/sFTP. That's got the benefit of being cross platform, but it would mean integrating a client into Calibre. The paths would be similar in that case, just drop /Volumes/root from the full path.
ldolse is offline   Reply With Quote
Old 06-02-2009, 10:27 AM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,991
Karma: 5036765
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
The first hurdle is device detection. How will calibre know when the iPhone is connected to the computer?
kovidgoyal is offline   Reply With Quote
Old 06-02-2009, 11:41 AM   #8
ldolse
Wizard
ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.
 
Posts: 1,337
Karma: 123455
Join Date: Apr 2009
Location: Malaysia
Device: PRS-650, iPhone
Device detection was one of the stumbling blocks I was hitting, especially as it doesn't mount as a disk. That's what got me thinking along the lines of accessing over the network instead.

Here's the best description I've found so far:
http://wikee.iphwn.org/usb:usbmux

Here are some open source projects that have some detection routines along with filesystem manipulation that can be checked out:
http://code.google.com/p/manzana/
http://code.google.com/p/independence/

I'll keep digging.
ldolse is offline   Reply With Quote
Old 06-02-2009, 11:50 AM   #9
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,991
Karma: 5036765
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
calibre's current infrastructure is geared towards managing USB devices, detecting a networked filesystem is a whole other beastie.
kovidgoyal is offline   Reply With Quote
Old 06-02-2009, 02:25 PM   #10
donCalpe
Junior Member
donCalpe began at the beginning.
 
Posts: 9
Karma: 10
Join Date: Apr 2008
Device: iRex DR1000S
As Idolse already mentioned, with a jailbroken iPhone and Netatalk installed ( under Mac OS X ) your iPhone mounts like any another computer ( it IS a computer ) you have in your network. You can use the finder to search for books ( i.e. ext. ".prc" extensions ). You can copy,transfer, move etc.
Regarding device detection - I add my books fromthe iPhone to the Calibre over the wireless network since I installed Calibre.
Idolse, if you want to automate some of the steps using Automator or Applescript is really straight forward.
The only thing left is the missing plug-in to deDRM your books.
donCalpe is offline   Reply With Quote
Old 06-20-2009, 02:02 AM   #11
tradewinds
Junior Member
tradewinds began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Jun 2009
Device: none
iphone kindle extraction

I got tired of all the manual steps, so wrote a little os x application to help me extract the mobi ebooks I've purchased using the iphone kindle app. Hope this helps someone else. Basically, it scans your iphone backup directory for mobi files, and decrypts them to a directory of your choice. You just have to add your iphone ID which you can get from itunes by clicking on serial # on Summary page for your iphone. Once extracted, you can then drag the mobi file into calibre.

search github for TradeWinds, dmg available on downloads tab

Last edited by tradewinds; 06-20-2009 at 02:04 AM. Reason: removed direct url
tradewinds is offline   Reply With Quote
Old 06-20-2009, 04:43 AM   #12
JSWolf
Resident Curmudgeon
JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.
 
JSWolf's Avatar
 
Posts: 37,284
Karma: 18156082
Join Date: Nov 2006
Location: Roslindale, Massachusetts
Device: Sony Reader PRS-650, iPad, nook STR
Quote:
Originally Posted by tradewinds View Post
I got tired of all the manual steps, so wrote a little os x application to help me extract the mobi ebooks I've purchased using the iphone kindle app. Hope this helps someone else. Basically, it scans your iphone backup directory for mobi files, and decrypts them to a directory of your choice. You just have to add your iphone ID which you can get from itunes by clicking on serial # on Summary page for your iphone. Once extracted, you can then drag the mobi file into calibre.

search github for TradeWinds, dmg available on downloads tab
What language did you write this application in? If you use Python or Pearl, it could then be cross platform compatible and also work on Windows.
JSWolf is offline   Reply With Quote
Old 06-20-2009, 11:34 AM   #13
tradewinds
Junior Member
tradewinds began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Jun 2009
Device: none
Quote:
Originally Posted by JSWolf View Post
What language did you write this application in? If you use Python or Pearl, it could then be cross platform compatible and also work on Windows.
The logic is in ruby, so thats cross platform, but all it really is is a glue layer between apple's interface builder and kindelpid/mobidedrm, so not really portable as is.
tradewinds is offline   Reply With Quote
Old 06-20-2009, 07:52 PM   #14
ldolse
Wizard
ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.
 
Posts: 1,337
Karma: 123455
Join Date: Apr 2009
Location: Malaysia
Device: PRS-650, iPhone
Awesome idea! Now if I can only get it to work. GUI itself is straightforward, but there seems to be a path issue with Python. I can browse the ebooks in the backup, but as soon as I attempt to decrypt the books I get this error:
Code:
Unable to get PID from iphone id: sh: python:
Command not found
I do have python of course, Here is the path and permissions info for the python that's active on my system:

Code:
$ which python
/usr/bin/python
$ ls -l /usr/bin/python
lrwxr-xr-x  1 root  wheel  72 Feb 22  2008 /usr/bin/python -> ../../System/Library/Frameworks/Python.framework/Versions/2.5/bin/python

One bit of feedback on the GUI - would be nice to have date as a sort option, sorted by date with most recent on top. Not required though, easy enough to find the latest book as the titles are there.
ldolse is offline   Reply With Quote
Old 06-20-2009, 09:09 PM   #15
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,058
Karma: 777825
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
What makes you think you cannot sort on the Data field? I regularily do it to find the most recently added titles.
itimpi is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
All new Nook app for IPhone and IPod touch boswd News 1 08-17-2010 12:37 PM
How well does the Kindle app work on the iPod Touch? shortywriter Apple Devices 2 06-13-2010 05:54 PM
Sunrise XP-like app for iPhone/iPod Touch? wildeny Apple Devices 2 01-17-2010 11:17 PM
Kindle for iPhone/iPod Touch goes International daffy4u News 1 12-14-2009 06:20 PM
Kindle app for iPhone and iPod Touch in iTunes Store cerement News 303 10-26-2009 05:39 PM


All times are GMT -4. The time now is 02:19 PM.


MobileRead.com is a privately owned, operated and funded community.