Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Readers > Amazon Kindle > Kindle Fire

Notices

Reply
 
Thread Tools Search this Thread
Old 03-05-2012, 02:16 AM   #1
jddunn
Junior Member
jddunn began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Mar 2012
Device: Kindle Fire
Sideloading + Annotations and Highlights Workflows?

Hello all. I've been reading here since I got my Fire last Dec., trying to get an idea of how things work on this platform and figure out how to set things up for the long haul.

If I'm going to use this as my main reading device, I need to get some stuff resolved and get a workflow in place for adding, archiving, and managing books, and especially for dealing with annotations and highlights.

I want to be able to use a mix of Amazon-purchased and sideloaded public domain books, as seamlessly as I can. From what I can tell, that rules out Whispernet and the Cloud Drive, which is pretty much fine with me. It'd be nice, and I thought about messing around with metadata hacks to fool Whispernet and such, but it looked inconsistent and like just too much trouble in the long run.

So, the answer is Calibre, right? Manage and archive my library with that, convert any new Amazon books that I purchase to the same format as my sideloaded books (probably Mobi?), and go onward from there.

But, what about the annotations and highlights and such? Is there a way to reliably manage those, export and import them so they aren't tied to this device forever, make sure they stay linked to the appropriate locations in the book file, and so on? I found the experimental "fetch annotations" feature in Calibre, but it's not working for me in tests with existing annotated mobi books, and even if I get it working, it's not clear from the documentation whether it keeps the annotations tied to the appropriate place in the book file, or whether it just spits them out in plain text and appends them to the file as a comment.

I'm currently in grad school and have to read a lot of PDFs for classes, and I've been able to get this sort of workflow working with those through a combination of RepliGo Reader and Dropbox. I keep the PDFs in my Dropbox folder, download them to the Kindle when I'm ready to read them, make highlights and notes in RepliGo, save, and Dropbox syncs the changes back to my PC. But, PDF's are self-contained, so that's probably a simpler nut to crack.

Is there a more general reader program like Mantano that does this kind of thing with ebooks? That, or are there hacks in the pipeline to make the Amazon reader work with things like Dropbox, or handle annotations and highlights consistently and make them exportable?

I read something about how the people initially rooting the Fire saw a lot of potential for extending the Amazon reader, but I haven't found any concrete followup on that so far. If that stuff is coming, I'd really rather stick with the Amazon reader because I like the interface and the added things like the dictionary and Wikipedia lookup, but I'm willing to look elsewhere if I have to.

Any tips on how to get this all working in a sustainable and consistent way that's not going to give me lots of headaches from managing sideloaded files separately from Amazon content, or trap all of my stuff on the same device or platform forever?
jddunn is offline   Reply With Quote
Old 11-04-2012, 10:57 AM   #2
Tracy D.
N00b. Dumb but fearless.
Tracy D. juggles running chainsaws for a bit of light exerciseTracy D. juggles running chainsaws for a bit of light exerciseTracy D. juggles running chainsaws for a bit of light exerciseTracy D. juggles running chainsaws for a bit of light exerciseTracy D. juggles running chainsaws for a bit of light exerciseTracy D. juggles running chainsaws for a bit of light exerciseTracy D. juggles running chainsaws for a bit of light exerciseTracy D. juggles running chainsaws for a bit of light exerciseTracy D. juggles running chainsaws for a bit of light exerciseTracy D. juggles running chainsaws for a bit of light exerciseTracy D. juggles running chainsaws for a bit of light exercise
 
Tracy D.'s Avatar
 
Posts: 7
Karma: 38508
Join Date: Sep 2011
Device: K4, Touch, PW
What a great question. If this is addressed in other threads, I haven't found them (searching for Kindle, cloud, annotations, et al). Would welcome any pointers.
Tracy D. is offline   Reply With Quote
Old 12-12-2012, 11:18 AM   #3
jddunn
Junior Member
jddunn began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Mar 2012
Device: Kindle Fire
Workflow Update

For the curious, here is the setup I finally figured out to cover most of these needs. It is complicated and combines 4 different tools:

1. Calibre for managing the sideloaded books and uploading them to the Fire.
2. Dropbox as background infrastructure for syncing things up
3. Dropsync to do the actual syncing.
4. The Kindle app on the PC, and the built in Kindle Reader on the Fire.

But, once you get it set up, you can pretty much leave it alone and it takes care of itself.

First, I created a folder named "Books" (but any name will work fine) in my Dropbox folder on the PC. Then I went into the Kindle app on the PC and changed the default folder it uses to store synced books, under Tools - Options - Content, to that new Dropbox folder.

Next, I installed Dropbox and Dropsync on my Fire, and got Dropbox connected up with my account. These aren't in the Amazon store, but both have free versions and can be sideloaded without rooting the Fire. Look around elsewhere on this forum for how to find .APK files and install them on the Fire if you're unsure.

Third, I imported the books I intend to sideload into Calibre on the PC, added cover images and edited metadata and such, and then converted them to .mobi format using the Kindle Fire profile. I also changed the Personal Document Tag from PDOC to EBOK in the process, which ensures that the books will show up under "Books" rather than "Docs" on the Fire. This can be found under the "Mobi Output" tab in the conversion dialogue. Even if you already have a .mobi file, you may want or need to reconvert it in Calibre just to change that tag. I haven't noticed any real formatting problems as a result of doing this.

With the books to be sideloaded fully ready, I now plug the Fire into the PC with a USB cable and upload them to the device using Calibre. This deposits them in the /Documents folder on the Fire, under the /sdcard folder in the Fire's file tree.

This is where Dropsync comes in. Dropsync allows you to automatically sync up the contents of a folder on your Fire with a folder in your Dropbox, so any changes to files on the Fire will automatically get uploaded to your Dropbox, and thus any computers you are running Dropbox on. The free version only lets you do this for one folder at a time, but that's all you happen to need for this, so you're good to go.

Using Dropsync, I linked up the /Documents folder on the Fire with the Books folder I had created earlier in my Dropbox. There are a few options on how to do this, but I set this syncing to be upload-only, so that only changes made on the Fire are synced up (more on this in a moment.)

Opening one of the sideloaded books up in Kindle Reader on the Fire to test this should result in the creation of .mbp and .azw files for that book, which Dropsync will discover and sync to your Dropbox. These are the files that store your annotations, bookmarks, and progress in the book.

This can be verified by opening up the Kindle app on the PC, which should now have the sideloaded books, with any annotations you made on the Fire. Those annotations will now be updated every time you make a change on the Fire in the future.

Caveats:

1. This setup is only handling syncing for sideloaded books. Whispersync and the Kindle app are still taking care of your Amazon-purchased books. This is why you set Dropsync to upload-only, because you don't want it grabbing all of those Whispersynced files sitting in the Books folder in your Dropbox and then creating duplicate books on the Fire under /Documents.

2. As a result, this is only doing 1-way syncing of annotations and progress, which backs up the changes you make on the Fire. Any changes you make in the Kindle app on the PC will not propagate back to the Fire, and will be overwritten if/when you make further changes on the Fire and Dropsync syncs them. This is more of an archiving / mirroring solution than a full, seamless multi-device setup. But, it does get your annotations off of the Fire and backed up to your PC, and lets you keep them there and review them for future reference, which was the most important thing for me.

3. I don't use Personal Documents and the email conversion service, so I haven't tested that case. If you do, there might be some duplication or conflict issues there, as that stuff all goes in the /Documents folder on the Fire as well, and you might then have both Whispersync and Dropsync trying to sync those files.

4. I haven't tested this on the Mac, but all of the necessary software is available on there, and there's no reason it shouldn't work. You might run into trouble with duplicates and overwritten changes if you run Dropbox and the Kindle app on multiple computers simultaneously though, as then you'll have both Whispersync and Dropbox syncing files in that folder on different machines, and I'm not sure what the results would be.

5. I also haven't tested this on the Fire HD. It should still work in principle, but things might change depending on whether it changes where things are stored and how organization of books is handled. The main requirement for this to work is that Whispersync and sideloaded books end up in different folders in the Fire's file tree, so if that still holds, you should be able to get it working.

Last edited by jddunn; 12-12-2012 at 11:26 AM.
jddunn is offline   Reply With Quote
Old 12-12-2012, 05:17 PM   #4
jswinden
Astrophotographer
jswinden ought to be getting tired of karma fortunes by now.jswinden ought to be getting tired of karma fortunes by now.jswinden ought to be getting tired of karma fortunes by now.jswinden ought to be getting tired of karma fortunes by now.jswinden ought to be getting tired of karma fortunes by now.jswinden ought to be getting tired of karma fortunes by now.jswinden ought to be getting tired of karma fortunes by now.jswinden ought to be getting tired of karma fortunes by now.jswinden ought to be getting tired of karma fortunes by now.jswinden ought to be getting tired of karma fortunes by now.jswinden ought to be getting tired of karma fortunes by now.
 
jswinden's Avatar
 
Posts: 5,427
Karma: 6850882
Join Date: Sep 2006
Location: USA
Device: iPad Mini 2, iPhone 5, Nexus 7.2
Quote:
Originally Posted by jddunn View Post
For the curious, here is the setup I finally figured out to cover most of these needs. It is complicated and combines 4 different tools:

1. Calibre for managing the sideloaded books and uploading them to the Fire.
2. Dropbox as background infrastructure for syncing things up
3. Dropsync to do the actual syncing.
4. The Kindle app on the PC, and the built in Kindle Reader on the Fire.

But, once you get it set up, you can pretty much leave it alone and it takes care of itself.

First, I created a folder named "Books" (but any name will work fine) in my Dropbox folder on the PC. Then I went into the Kindle app on the PC and changed the default folder it uses to store synced books, under Tools - Options - Content, to that new Dropbox folder.

Next, I installed Dropbox and Dropsync on my Fire, and got Dropbox connected up with my account. These aren't in the Amazon store, but both have free versions and can be sideloaded without rooting the Fire. Look around elsewhere on this forum for how to find .APK files and install them on the Fire if you're unsure.

Third, I imported the books I intend to sideload into Calibre on the PC, added cover images and edited metadata and such, and then converted them to .mobi format using the Kindle Fire profile. I also changed the Personal Document Tag from PDOC to EBOK in the process, which ensures that the books will show up under "Books" rather than "Docs" on the Fire. This can be found under the "Mobi Output" tab in the conversion dialogue. Even if you already have a .mobi file, you may want or need to reconvert it in Calibre just to change that tag. I haven't noticed any real formatting problems as a result of doing this.

With the books to be sideloaded fully ready, I now plug the Fire into the PC with a USB cable and upload them to the device using Calibre. This deposits them in the /Documents folder on the Fire, under the /sdcard folder in the Fire's file tree.

This is where Dropsync comes in. Dropsync allows you to automatically sync up the contents of a folder on your Fire with a folder in your Dropbox, so any changes to files on the Fire will automatically get uploaded to your Dropbox, and thus any computers you are running Dropbox on. The free version only lets you do this for one folder at a time, but that's all you happen to need for this, so you're good to go.

Using Dropsync, I linked up the /Documents folder on the Fire with the Books folder I had created earlier in my Dropbox. There are a few options on how to do this, but I set this syncing to be upload-only, so that only changes made on the Fire are synced up (more on this in a moment.)

Opening one of the sideloaded books up in Kindle Reader on the Fire to test this should result in the creation of .mbp and .azw files for that book, which Dropsync will discover and sync to your Dropbox. These are the files that store your annotations, bookmarks, and progress in the book.

This can be verified by opening up the Kindle app on the PC, which should now have the sideloaded books, with any annotations you made on the Fire. Those annotations will now be updated every time you make a change on the Fire in the future.

Caveats:

1. This setup is only handling syncing for sideloaded books. Whispersync and the Kindle app are still taking care of your Amazon-purchased books. This is why you set Dropsync to upload-only, because you don't want it grabbing all of those Whispersynced files sitting in the Books folder in your Dropbox and then creating duplicate books on the Fire under /Documents.

2. As a result, this is only doing 1-way syncing of annotations and progress, which backs up the changes you make on the Fire. Any changes you make in the Kindle app on the PC will not propagate back to the Fire, and will be overwritten if/when you make further changes on the Fire and Dropsync syncs them. This is more of an archiving / mirroring solution than a full, seamless multi-device setup. But, it does get your annotations off of the Fire and backed up to your PC, and lets you keep them there and review them for future reference, which was the most important thing for me.

3. I don't use Personal Documents and the email conversion service, so I haven't tested that case. If you do, there might be some duplication or conflict issues there, as that stuff all goes in the /Documents folder on the Fire as well, and you might then have both Whispersync and Dropsync trying to sync those files.

4. I haven't tested this on the Mac, but all of the necessary software is available on there, and there's no reason it shouldn't work. You might run into trouble with duplicates and overwritten changes if you run Dropbox and the Kindle app on multiple computers simultaneously though, as then you'll have both Whispersync and Dropbox syncing files in that folder on different machines, and I'm not sure what the results would be.

5. I also haven't tested this on the Fire HD. It should still work in principle, but things might change depending on whether it changes where things are stored and how organization of books is handled. The main requirement for this to work is that Whispersync and sideloaded books end up in different folders in the Fire's file tree, so if that still holds, you should be able to get it working.
Personal docs sent to the Amazon Cloud will allow Whispersyncing of bookmarks, highlighting, notes, and farthest page read to most devices and apps. No need to go through such a complicated/complex set up.
jswinden is online now   Reply With Quote
Old 12-12-2012, 10:28 PM   #5
jddunn
Junior Member
jddunn began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Mar 2012
Device: Kindle Fire
Well, I wanted control over the formatting and metadata of my sideloaded books, an integrated library with all of the books showing up under the books section on the Fire, the ability to get the annotations out of Amazon and move to another reader if and when I want to, and so on. I'm also not crazy about Amazon having access to all of my stuff that I didn't buy from them. If you don't want that sort of control, Pdocs are fine. If you do, and don't mind the tradeoffs, then here's how to get it.
jddunn is offline   Reply With Quote
Old 12-13-2012, 03:59 AM   #6
beckywc
Addict
beckywc ought to be getting tired of karma fortunes by now.beckywc ought to be getting tired of karma fortunes by now.beckywc ought to be getting tired of karma fortunes by now.beckywc ought to be getting tired of karma fortunes by now.beckywc ought to be getting tired of karma fortunes by now.beckywc ought to be getting tired of karma fortunes by now.beckywc ought to be getting tired of karma fortunes by now.beckywc ought to be getting tired of karma fortunes by now.beckywc ought to be getting tired of karma fortunes by now.beckywc ought to be getting tired of karma fortunes by now.beckywc ought to be getting tired of karma fortunes by now.
 
beckywc's Avatar
 
Posts: 270
Karma: 1050426
Join Date: Feb 2010
Location: California
Device: iPad Mini w/Retina, Kindle 3, Kindle Fire HDX 8.9, & Asus Transformer
Quote:
Originally Posted by jddunn View Post
Well, I wanted control over the formatting and metadata of my sideloaded books, an integrated library with all of the books showing up under the books section on the Fire, the ability to get the annotations out of Amazon and move to another reader if and when I want to, and so on. I'm also not crazy about Amazon having access to all of my stuff that I didn't buy from them. If you don't want that sort of control, Pdocs are fine. If you do, and don't mind the tradeoffs, then here's how to get it.
I agree with you I want control as well. The only thing I do differently is that I have Calibre send my books to my book directory instead of documents directory.
beckywc is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Viewing annotations/highlights from Dx on pc for editing purpose files JustBrowsing Amazon Kindle 3 02-06-2012 09:43 AM
DRM and popular-highlights/annotations granturissimus Amazon Kindle 2 11-04-2011 06:06 AM
Sideloading Annotations and Highlights professorbeej Amazon Kindle 2 08-17-2011 04:27 AM
Does Calibre have the ability to do highlights & annotations? mobi555 Library Management 0 06-22-2011 12:36 AM
PRS-600 Restore deleted annotations and highlights facruz Sony Reader 0 01-29-2011 11:10 PM


All times are GMT -4. The time now is 12:00 PM.


MobileRead.com is a privately owned, operated and funded community.