MobileRead Forums - View Single Post - Sideloading + Annotations and Highlights Workflows?

jddunn · 12-12-2012, 11:18 AM

For the curious, here is the setup I finally figured out to cover most of these needs. It is complicated and combines 4 different tools:

1. Calibre for managing the sideloaded books and uploading them to the Fire.
2. Dropbox as background infrastructure for syncing things up
3. Dropsync to do the actual syncing.
4. The Kindle app on the PC, and the built in Kindle Reader on the Fire.

But, once you get it set up, you can pretty much leave it alone and it takes care of itself.

First, I created a folder named "Books" (but any name will work fine) in my Dropbox folder on the PC. Then I went into the Kindle app on the PC and changed the default folder it uses to store synced books, under Tools - Options - Content, to that new Dropbox folder.

Next, I installed Dropbox and Dropsync on my Fire, and got Dropbox connected up with my account. These aren't in the Amazon store, but both have free versions and can be sideloaded without rooting the Fire. Look around elsewhere on this forum for how to find .APK files and install them on the Fire if you're unsure.

Third, I imported the books I intend to sideload into Calibre on the PC, added cover images and edited metadata and such, and then converted them to .mobi format using the Kindle Fire profile. I also changed the Personal Document Tag from PDOC to EBOK in the process, which ensures that the books will show up under "Books" rather than "Docs" on the Fire. This can be found under the "Mobi Output" tab in the conversion dialogue. Even if you already have a .mobi file, you may want or need to reconvert it in Calibre just to change that tag. I haven't noticed any real formatting problems as a result of doing this.

With the books to be sideloaded fully ready, I now plug the Fire into the PC with a USB cable and upload them to the device using Calibre. This deposits them in the /Documents folder on the Fire, under the /sdcard folder in the Fire's file tree.

This is where Dropsync comes in. Dropsync allows you to automatically sync up the contents of a folder on your Fire with a folder in your Dropbox, so any changes to files on the Fire will automatically get uploaded to your Dropbox, and thus any computers you are running Dropbox on. The free version only lets you do this for one folder at a time, but that's all you happen to need for this, so you're good to go.

Using Dropsync, I linked up the /Documents folder on the Fire with the Books folder I had created earlier in my Dropbox. There are a few options on how to do this, but I set this syncing to be upload-only, so that only changes made on the Fire are synced up (more on this in a moment.)

Opening one of the sideloaded books up in Kindle Reader on the Fire to test this should result in the creation of .mbp and .azw files for that book, which Dropsync will discover and sync to your Dropbox. These are the files that store your annotations, bookmarks, and progress in the book.

This can be verified by opening up the Kindle app on the PC, which should now have the sideloaded books, with any annotations you made on the Fire. Those annotations will now be updated every time you make a change on the Fire in the future.

Caveats:

1. This setup is only handling syncing for sideloaded books. Whispersync and the Kindle app are still taking care of your Amazon-purchased books. This is why you set Dropsync to upload-only, because you don't want it grabbing all of those Whispersynced files sitting in the Books folder in your Dropbox and then creating duplicate books on the Fire under /Documents.

2. As a result, this is only doing 1-way syncing of annotations and progress, which backs up the changes you make on the Fire. Any changes you make in the Kindle app on the PC will not propagate back to the Fire, and will be overwritten if/when you make further changes on the Fire and Dropsync syncs them. This is more of an archiving / mirroring solution than a full, seamless multi-device setup. But, it does get your annotations off of the Fire and backed up to your PC, and lets you keep them there and review them for future reference, which was the most important thing for me.

3. I don't use Personal Documents and the email conversion service, so I haven't tested that case. If you do, there might be some duplication or conflict issues there, as that stuff all goes in the /Documents folder on the Fire as well, and you might then have both Whispersync and Dropsync trying to sync those files.

4. I haven't tested this on the Mac, but all of the necessary software is available on there, and there's no reason it shouldn't work. You might run into trouble with duplicates and overwritten changes if you run Dropbox and the Kindle app on multiple computers simultaneously though, as then you'll have both Whispersync and Dropbox syncing files in that folder on different machines, and I'm not sure what the results would be.

5. I also haven't tested this on the Fire HD. It should still work in principle, but things might change depending on whether it changes where things are stored and how organization of books is handled. The main requirement for this to work is that Whispersync and sideloaded books end up in different folders in the Fire's file tree, so if that still holds, you should be able to get it working.

12-12-2012, 11:18 AM	#3
jddunn Junior Member Posts: 3 Karma: 10 Join Date: Mar 2012 Device: Kindle Fire	Workflow Update For the curious, here is the setup I finally figured out to cover most of these needs. It is complicated and combines 4 different tools: 1. Calibre for managing the sideloaded books and uploading them to the Fire. 2. Dropbox as background infrastructure for syncing things up 3. Dropsync to do the actual syncing. 4. The Kindle app on the PC, and the built in Kindle Reader on the Fire. But, once you get it set up, you can pretty much leave it alone and it takes care of itself. First, I created a folder named "Books" (but any name will work fine) in my Dropbox folder on the PC. Then I went into the Kindle app on the PC and changed the default folder it uses to store synced books, under Tools - Options - Content, to that new Dropbox folder. Next, I installed Dropbox and Dropsync on my Fire, and got Dropbox connected up with my account. These aren't in the Amazon store, but both have free versions and can be sideloaded without rooting the Fire. Look around elsewhere on this forum for how to find .APK files and install them on the Fire if you're unsure. Third, I imported the books I intend to sideload into Calibre on the PC, added cover images and edited metadata and such, and then converted them to .mobi format using the Kindle Fire profile. I also changed the Personal Document Tag from PDOC to EBOK in the process, which ensures that the books will show up under "Books" rather than "Docs" on the Fire. This can be found under the "Mobi Output" tab in the conversion dialogue. Even if you already have a .mobi file, you may want or need to reconvert it in Calibre just to change that tag. I haven't noticed any real formatting problems as a result of doing this. With the books to be sideloaded fully ready, I now plug the Fire into the PC with a USB cable and upload them to the device using Calibre. This deposits them in the /Documents folder on the Fire, under the /sdcard folder in the Fire's file tree. This is where Dropsync comes in. Dropsync allows you to automatically sync up the contents of a folder on your Fire with a folder in your Dropbox, so any changes to files on the Fire will automatically get uploaded to your Dropbox, and thus any computers you are running Dropbox on. The free version only lets you do this for one folder at a time, but that's all you happen to need for this, so you're good to go. Using Dropsync, I linked up the /Documents folder on the Fire with the Books folder I had created earlier in my Dropbox. There are a few options on how to do this, but I set this syncing to be upload-only, so that only changes made on the Fire are synced up (more on this in a moment.) Opening one of the sideloaded books up in Kindle Reader on the Fire to test this should result in the creation of .mbp and .azw files for that book, which Dropsync will discover and sync to your Dropbox. These are the files that store your annotations, bookmarks, and progress in the book. This can be verified by opening up the Kindle app on the PC, which should now have the sideloaded books, with any annotations you made on the Fire. Those annotations will now be updated every time you make a change on the Fire in the future. Caveats: 1. This setup is only handling syncing for sideloaded books. Whispersync and the Kindle app are still taking care of your Amazon-purchased books. This is why you set Dropsync to upload-only, because you don't want it grabbing all of those Whispersynced files sitting in the Books folder in your Dropbox and then creating duplicate books on the Fire under /Documents. 2. As a result, this is only doing 1-way syncing of annotations and progress, which backs up the changes you make on the Fire. Any changes you make in the Kindle app on the PC will not propagate back to the Fire, and will be overwritten if/when you make further changes on the Fire and Dropsync syncs them. This is more of an archiving / mirroring solution than a full, seamless multi-device setup. But, it does get your annotations off of the Fire and backed up to your PC, and lets you keep them there and review them for future reference, which was the most important thing for me. 3. I don't use Personal Documents and the email conversion service, so I haven't tested that case. If you do, there might be some duplication or conflict issues there, as that stuff all goes in the /Documents folder on the Fire as well, and you might then have both Whispersync and Dropsync trying to sync those files. 4. I haven't tested this on the Mac, but all of the necessary software is available on there, and there's no reason it shouldn't work. You might run into trouble with duplicates and overwritten changes if you run Dropbox and the Kindle app on multiple computers simultaneously though, as then you'll have both Whispersync and Dropbox syncing files in that folder on different machines, and I'm not sure what the results would be. 5. I also haven't tested this on the Fire HD. It should still work in principle, but things might change depending on whether it changes where things are stored and how organization of books is handled. The main requirement for this to work is that Whispersync and sideloaded books end up in different folders in the Fire's file tree, so if that still holds, you should be able to get it working. Last edited by jddunn; 12-12-2012 at 11:26 AM.