Creating Kindle Collections from Calibre's Data

[Perkin]

the print lines would now be: (your line 54 only)

print ("[[PHASE: CREATING MISCELLANY COLLECTIONS]]\n")

[MrTissues]

Originally Posted by Perkin

the print lines would now be: (your line 54 only)

print ("[[PHASE: CREATING MISCELLANY COLLECTIONS]]\n")

Thank you for your help. I think I am just going to use 2.7, it did work when I made that switch.

EDIT: I just got my Kindle and Collections do not seem to work how I thought they would. Is it not possible to switch between collection view and a view that just dumps all the books?

I was hoping to have collections for my book series but still be able to find all of my books that do not have a series in the "non collection" view. Is this not possible?

[LabRat3]

I'm having a weird problem, every time I try to load either version of the script I'm getting an import error where it says no module named json. I feel like there is probably an obvious solution to this problem, but it eludes me. Could it be because the system file is hidden possibly? (I checked, the json file is there).

Thanks for any help. Oh, and its Kindle 3 from 64bit Windows 7 with Python 2.7. I tried using admin command line as well to no avail.

[MrTissues]

Okay I figured out my above question. One last final question. When importing a series is it possible to add a number such as 1 or 2 in front of the title so that they will be forced to be in correct series order? It would be best if I could keep the names the same in the general view but only have the numbers in the collections.

[mornington]

First off - the script's written under Python 2.7 (as I note in the installation, erm, notes). It won't work under Python 3, which apparently breaks all backwards compatibility.

Secondly, I'm afraid I can't help with the script at all anymore as I no longer own a Kindle: Basically, I got so annoyed by all the various problems with the Kindle's interface (like collections management, which I wrote this script to solve, but also the constantly changing and unreliable Kindle date/time setting; the annoying and almost constant indexing of books, which can't be reliably switched off permanently; the very sluggish response when more than about 1200 books are loaded onto it, etc, etc)...

So, basically, I sent the Kindle back, got a refund and have now bought a Sony PRS-650, which I'm much happier with.

This does mean, however, that I'm unable to help any more with extending/refining the script.

[mornington]

Originally Posted by MrTissues

Okay I figured out my above question. One last final question. When importing a series is it possible to add a number such as 1 or 2 in front of the title so that they will be forced to be in correct series order? It would be best if I could keep the names the same in the general view but only have the numbers in the collections.

Short answer is no - script only handles collections, not book titles.

However, look into calibre's metadata plugboards, as those can do this for you. But there are some Kindle-specific pitfalls along the way, so you may need to experiment a bit.

[mornington]

Originally Posted by MrTissues

Thank you for your help. I think I am just going to use 2.7, it did work when I made that switch.

EDIT: I just got my Kindle and Collections do not seem to work how I thought they would. Is it not possible to switch between collection view and a view that just dumps all the books?

I was hoping to have collections for my book series but still be able to find all of my books that do not have a series in the "non collection" view. Is this not possible?

Not as such, no. The "by title" and "by author" views show all books *and all collections* and the "by collections" view shows all collections and all books which are not in any collection. Or the script will, by default, create a "- Miscellany" collection which contains all books which are not in any other collection.

[anesthezea]

Originally Posted by mornington

Secondly, I'm afraid I can't help with the script at all anymore as I no longer own a Kindle: Basically, I got so annoyed by all the various problems with the Kindle's interface (like collections management, which I wrote this script to solve, but also the constantly changing and unreliable Kindle date/time setting; the annoying and almost constant indexing of books, which can't be reliably switched off permanently; the very sluggish response when more than about 1200 books are loaded onto it, etc, etc)...

So, basically, I sent the Kindle back, got a refund and have now bought a Sony PRS-650, which I'm much happier with.

I ordered a K3 when it was first released and after a month of doing more collection sorting and pulling of hair than actual reading, I sold it and bought the Sony 650. I received another K3 for Christmas, so I decided to give it another try.

I've had the thing for less than 5 days now and I'm still struggling to get the thing organized the way I want it. Again, I'm spending most of my time up to my elbows in Calibre, plugboards, and Python. I have yet to actually READ on the darn thing!

I'm going back to my Sony 650. While the K3 was a lovely gift, it's proving once again that it's just not the device for me.

[Jankomatic]

Is there an easy way to remove all the things the script does for collections, so back out changes it made.

Thanks

[CWatkinsNash]

Originally Posted by Jankomatic

Is there an easy way to remove all the things the script does for collections, so back out changes it made.

Thanks

If you haven't used the script yet, you can make a backup copy of your current collections.json file and keep it on your computer. To restore, just copy it back to the Kindle System folder and restart.

If you've already used the script and don't like the results and have no backup, one quick but customizable way to get rid of the collections is by installing Kindle Collection Manager. There is more info about that app in this thread. If you use it, you can either delete all of the collections, or keep the ones you want as well as making changes to them.

You can also use Kindle Collection Manager to manage your collections by itself, but I like the functionality of the script so I've been using it instead.

[FF2]

There's also this method:

http://www.accwebworks.com/kindle.aspx

which does rely on your creating a text file containing your kindle's folder structure. I found it easy to use. You have to trust that the web site is not going to do anything odd to your text file but the file it generates is a collections.json file which you can read for safety.

[Jankomatic]

The script does not work properly for me for some reason. It did not compile any collections correctly. I have multiple series with 5+ books in them and it makes a series and usually puts half of them in that collection and then creates a different collection for the other half. I have verified multiple times that they are set up correctly in Calibre first. I am not using TAGS and have author collections turned off on the command line. So not sure why it is messing up.

[CWatkinsNash]

Originally Posted by Jankomatic

The script does not work properly for me for some reason. It did not compile any collections correctly. I have multiple series with 5+ books in them and it makes a series and usually puts half of them in that collection and then creates a different collection for the other half. I have verified multiple times that they are set up correctly in Calibre first. I am not using TAGS and have author collections turned off on the command line. So not sure why it is messing up.

A few questions:
Computer operating system?
Python version?
Have you made any changes to the script?

Are you using logging? If not, add:

--verbose > CalibreKindleCollections.log

to the end of the command line string you are using. This will create and deposit a log file into the directory you run the script from (usually the Kindle root unless you're running it with a mount point from elsewhere). Attach that to your reply here and I'll take a look at it.

[Jankomatic]

Vista 64 bit
Python 2.7
No changes to the script, I am running it like this though:

CalibreKindleCollections.py --na --nt --verbose > CKC.log

Take a look at the first of the log file, you see a series call A Song of Fire and Ice, it has 4 books in it. After restarting my Kindle it only has 2 books in that series, the third and fourth ones. The other two books in the series appear in the list after all the collections are listed. Like I said it does all the collections like this, it puts seemingly random books from the series in the collection, then the rest just dumps at the end. Looking at the next section of the log, it looks like it is working correctly, but again on the Kindle it doesn't look like that.

I have deleted the json file and had it run from scratch multiple times and even run it with the --rb option, always seems the same except for one thing. It always seems to put the same number of books in the collection for example Song of Fire and Ice it always puts 2 books in that collection, but the 2 it "picks" are random. They aren't the same each time for any series it makes a collection for.

Thanks for taking a look.

[CWatkinsNash]

Originally Posted by Jankomatic

Vista 64 bit
Python 2.7
No changes to the script, I am running it like this though:

CalibreKindleCollections.py --na --nt --verbose > CKC.log

Take a look at the first of the log file, you see a series call A Song of Fire and Ice, it has 4 books in it. After restarting my Kindle it only has 2 books in that series, the third and fourth ones. The other two books in the series appear in the list after all the collections are listed. Like I said it does all the collections like this, it puts seemingly random books from the series in the collection, then the rest just dumps at the end. Looking at the next section of the log, it looks like it is working correctly, but again on the Kindle it doesn't look like that.

Thanks for taking a look.

I had this same problem with some books. I was starting to think I was the only one, but fortunately this means I can tell you how I fixed it. For some reason, it sometimes detects files as "dead" (no longer there) and the cleanup function pops them back out of the collections. Try this:

CalibreKindleCollections.py --rb --nc --na --nt --verbose > CKC.log

The --rb runs it as a rebuild (rebuilds all collections from scratch).
The --nc will stop it from cleaning up files it thinks are not there.

Eject, restart, and after 5-10 seconds or so, you should see all your collections the way they should be. After running it this time, for future updating you can leave the --rb off unless you really do need to rebuild them all.

After this, the only files that will still get kicked out are those with Amazon ASINs (with EBOK in the hash) but you only have a few of those and it doesn't look like they are in series anyway so that shouldn't be an issue. I just wanted to bring it up in case you do more collections later with tags or authors. There is a method mentioned somewhere around here for adding them to the script for handling but unless you have a lot, it's not worth the time, IMO.

Using the --nc option shouldn't cause any problems. I've been doing it this way for over a month and I've never had a problem with "ghost" deleted files showing up anywhere. If you do a lot of deleting books off the Kindle, though, you may want to run a rebuild once in a while to make sure the collections.json doesn't get leftover entries in it.

Let me know how this works for you. I have two bat files I use (one for rebuild, one for updating). I stick them in the Kindle root with the script so I just have to double click the one I want to use. Let me know if you want to see them them and I'll copy them over and attach - you'll just have to change the python path and switches you want to use.