uploading books to kindle cloud collections

[dhdurgee]

I have just purchased a Kindle Paperwhite 4 as a replacement for a Kindle Touch 2 that I lost a few weeks ago. I have jailbroken the PW4 and am now running customer firmware 5.10.1.3 as recommended by several other Kindle developer's forum users.

It appears that this firmware unlike the older release I was running on the KT2 has access to Kindle cloud collections, which it appears to have created from my collection on the KT2. Most of these collections are empty as I had sideloaded their contents to the KT2.

I have been trying to determine my options to upload my contents to the Kindle cloud collections, but don't seem to see an easy way to do so. There appear to be tools for Windows and android but not linux to upload books, but I am not sure if they permit uploading directly to a cloud collection.

It appears that I can upload from Calibre to the cloud via email, but once again can this be directed to a cloud collection or only to a device? If not, I guess I will need to either sideload books again or email them and then add them to the collections on the PW4.

I also just attempted to email some of my sideloaded books to my PW4 and received an error email for each one. These are all azw3 files, so I am puzzled by this. I guess I will contact Amazon and see what they have to say.

Any guidance will be appreciated.

Dave

[mergen3107]

So you cannot “upload” books to cloud per se, but you can “Send to Kindle” them (this is exactly what Calibre does by “sending to cloud”).
You need to go to your Amazon account from PC, go to Your Devices and Contents, and find there what is your Kindle-associated e-mail. There is also something like “a list of trusted e-mail addresses”, add your e-mail address which you going to send files from.
Then, you can send to this e-mail some formats like mobi, doc, pdf, and some others (I don’t remember) and they will appear on all your devices with the same Kindle account. However, IIRC all of them will have a label of “Personal Document”.

Regarding azw3, yes you cannot send them, unfortunately. It is related to ASIN number (a very long story), so in short try to convert them to .mobi files in Calibre. .mobi files are larger in size than azw3, because usually a .mobi files contains two formats: mobi7 and kf8 (which is azw3).

[dhdurgee]

Originally Posted by mergen3107

So you cannot “upload” books to cloud per se, but you can “Send to Kindle” them (this is exactly what Calibre does by “sending to cloud”).
You need to go to your Amazon account from PC, go to Your Devices and Contents, and find there what is your Kindle-associated e-mail. There is also something like “a list of trusted e-mail addresses”, add your e-mail address which you going to send files from.
Then, you can send to this e-mail some formats like mobi, doc, pdf, and some others (I don’t remember) and they will appear on all your devices with the same Kindle account. However, IIRC all of them will have a label of “Personal Document”.

Regarding azw3, yes you cannot send them, unfortunately. It is related to ASIN number (a very long story), so in short try to convert them to .mobi files in Calibre. .mobi files are larger in size than azw3, because usually a .mobi files contains two formats: mobi7 and kf8 (which is azw3).

I just tried converting a book to mobi and that was rejected as well. I should note that I had tweaked the conversion settings to output in the "new" format as opposed to "old" or "both" as that is supposed to be the kf8 version.

Ugh.

Dave

[ilovejedd]

Do you actually want your ebooks stored on the Amazon Cloud?

If not, since you're jailbroken, you can use the Kindle Collections plugin for Calibre in conjunction with LibrarianSync to create and add books to collections on the PW4.

Mind, I think you'd need either mobi "old" or "both" for uploading to Amazon. Alternately, rename epub to txt and email that to Amazon so they do the conversion themselves. Note, any books you email will be categorized as Docs rather than Books.

[dhdurgee]

Originally Posted by ilovejedd

Do you actually want your ebooks stored on the Amazon Cloud?

If not, since you're jailbroken, you can use the Kindle Collections plugin for Calibre in conjunction with LibrarianSync to create and add books to collections on the PW4.

Mind, I think you'd need either mobi "old" or "both" for uploading to Amazon. Alternately, rename epub to txt and email that to Amazon so they do the conversion themselves. Note, any books you email will be categorized as Docs rather than Books.

It doesn't look as if Amazon really wants me to use it, does it? Pehaps I should investigate using Librarian Sync and possibly the Kindle Collection plugin as well. While the PW4 is still pretty empty should be ideal.

If I understand this correctly I should simply be able to restructure my documents folder with sub-folders and then use Librarian Sync to create local collections. Will Librarian Sync work with all content, or only books? As I have two magazine subscriptions with Amazon will this interfere with that? Will I simply need to move the new issues to the proper folder and run Librarian Sync again?

Dave

[ilovejedd]

There are a few methods to create collections with Librarian Sync.

1. [/mnt/us/]extensions/collections.json: regex based on path and filenames, can have multiple collections per book
2. folders: based purely on folder hierarchy, one collection per book
3. [/mnt/us/]system/collections.json: generated by the Kindle Collections plugin for Calibre, can have multiple collections per book
   Note: system is a hidden system folder

Not sure how LS handles magazines. The Kindle I use with LS is unregistered as I prefer to keep my "normal" books separate from fics.

[NiLuJe]

I don't recall if there's a MAGZ/NWPR blacklist or special handling either, so, err, try it? .

It'll *probably* work. Or not . Not something I actively tested, in any case.

[dhdurgee]

Originally Posted by NiLuJe

I don't recall if there's a MAGZ/NWPR blacklist or special handling either, so, err, try it? .

It'll *probably* work. Or not . Not something I actively tested, in any case.

I installed Librarian Sync 20190103 on the PW4. I used the export option to get a sample .json for this PW4. I am now considering how to set this up.

Are nested categories supported using the .json file? If so, how are they entered? I am thinking something like a top level of category of book with author at the next level possibly with series below that holding the books. Since multiple collections can be specified that can obviously be done as well, but the top level categories would hold large numbers of entries and some authors are quite prolific.

Dave

[mergen3107]

Kindle itself doesn’t support nested collections.
All it has is top level and just one level of collection.

I am not sure whether LS can do multiple sub-collections for you, but at least you can do a set of collection-subcollections as folders and then use folder hierarchy to build collections in LS.

What I mean is:
If you currently have Author1/ folder with Series1/, Series2/, etc subfolders inside, you can create a folder “Author1 - Series1”, “Author1 - Series2”, etc folders.

P. S. All in all I dropped that idea of moving folders/subfolders structure in one-to-one correspondence to Kindle, because the indexing is really fast (in case when all books have finished indexing) and you can find any book in a matter of a few seconds … to be able to dive into it for a good 2 weeks or more

[ilovejedd]

Originally Posted by dhdurgee

I installed Librarian Sync 20190103 on the PW4. I used the export option to get a sample .json for this PW4. I am now considering how to set this up.

Are nested categories supported using the .json file? If so, how are they entered? I am thinking something like a top level of category of book with author at the next level possibly with series below that holding the books. Since multiple collections can be specified that can obviously be done as well, but the top level categories would hold large numbers of entries and some authors are quite prolific.

Nested categories are not supported in current Kindle firmware. Dunno when that change occurred but certainly by 5.6.x at least.

You can find a sample JSON at the bottom of this page:
https://github.com/NiLuJe/librariansync

If you're already using Calibre, I think the Kindle Collections plugin is the easiest way to generate the collections. You can easily create collections by authors, series, tags and even custom columns. Note, the format used in /extensions/collection.json is different from the format used by the Calibre plugin in /system/collections.json.

[dhdurgee]

Originally Posted by ilovejedd

Nested categories are not supported in current Kindle firmware. Dunno when that change occurred but certainly by 5.6.x at least.

You can find a sample JSON at the bottom of this page:
https://github.com/NiLuJe/librariansync

If you're already using Calibre, I think the Kindle Collections plugin is the easiest way to generate the collections. You can easily create collections by authors, series, tags and even custom columns. Note, the format used in /extensions/collection.json is different from the format used by the Calibre plugin in /system/collections.json.

Interesting. I would have expected nested folders to result in nested collections when collections were being built from folders. So what happens when nested folders are converted to collections?

I am using Calibre here, but I don't at present have everything in Calibre. I use Calibre in particular for converting formats, hyphenating books and removing DRM if necessary. I also used it to place books on the lost KT2 and can use it likewise with the PW4.

Until I sort out how best to organize collections I am delaying loading much content on the PW4.

Thanks again for your input in this matter.

Dave

[mergen3107]

Originally Posted by dhdurgee

So what happens when nested folders are converted to collections?

I guess, everything from that folder will be in one collection. I haven’t tried it this way myself, but I guess LS won’t leave anything behind during creating collections.

[NiLuJe]

Quoting the documentation: "Subfolders will be treated as different collections".

i.e., you can nest all you want, the file's first direct parent will dictate which collection it belongs to. AFAICT. Never used the folder method myself ;p.

[ilovejedd]

Originally Posted by dhdurgee

Interesting. I would have expected nested folders to result in nested collections when collections were being built from folders. So what happens when nested folders are converted to collections?

The full path (minus /mnt/us/documents/) becomes the collection name.

e.g.

/Fantasy/JK Rowling/Harry Potter/Harry Potter and the Philosopher's Stone.azw3
-> Collection: Fantasy/JK Rowling/Harry Potter

/Romance/Historical/Judith McNaught/Almost Heaven.azw3
-> Collection: Romance/Historical/Judith McNaught

/Romance/Suspense/JD Robb/In Death/Naked in Death.azw3
-> Collection: Romance/Thriller/JD Robb/In Death

I don't like to create collections for authors with only a few books though and I find pure folders to be too limited. I tend to use a combination of LibrarianSync JSON (for books downloaded via web browser or copied from phone/tablet) and Calibre JSON to generate collections. It works great as long as the collections don't overlap.

[ilovejedd]

P.S. It's trivially easy to rebuild collections via Librarian Sync. I scrapped my current collections to see how the folder method works. Afterwards, just ran the two JSON-based methods to get it back to the desired state.

2245 Items

1. Rebuild collections (from folders): 208 sec -> I think this created 600+ collections
2. Delete all collections: 64 sec
3. Rebuild collections (from calibre plugin): 40 sec -> 21 collections based on tags/custom columns
4. Update collections (librarian): 54 sec -> 34 collections mostly based on author names (favorites)

As an aside, I thought I only had half the number of books on this Kindle. No wonder full library text search is slow.