Issue with Calibre 0.7.1 & Kindle 2.5.2

[mornington]

I'm an idiot. The previous script was set to always completely rebuild the kindle collections (I'd left a debug setting in by accident).

The attached version works as advertised.

Also, I noticed that if you set calibre to "automatically manage" metadata, as suggested, then you may not be able to see your collections in the "device" window in calibre.

No idea why, but the Kindle can see them, and that's the important thing (and you can set either of the other metadata management options in calibre if you really want to see them there, but if you do then make sure you force calibre to refresh the metadata on your kindle, as described earlier, before running the script).

[durand]

Thanks for the script, it works really well! Is there a way to display all the files at once on the kindle? It seems to only show me collections now.

[mornington]

Originally Posted by durand

Thanks for the script, it works really well! Is there a way to display all the files at once on the kindle? It seems to only show me collections now.

When you're viewing in "sort by title" or "sort by author" you see everything - collections and books - all together. If you're using my script then the collections will have a dash in front of their names, so in "sort by title" the collections all appear first in the sorted list, followed by ALL the individual books.


Incidentally, the "clean up" parts of my script (the bits which look for dead files and empty collections) don't seem to be working brilliantly well, so those could probably do with the eye of somebody who actually knows python to figure out what's wrong there. They don't do anything harmful, though.

[mornington]

Also, bear in mind that the kindle can take a few minutes to place all the files in the right collections and populate the "home" page following a restart. There's nothing to be done about that, though.

Just give it time - it's slow, but it'll get there in the end.

[durand]

Oh so thats why you've added a dash :S I removed it because I didn't see a need for it. It seems to be working now. Thanks! I'll have a look at the cleanup bit in a week when university's out of the way.

[mornington]

Added a few notes to the head of the script file, explaining how to configure it.

Also, I noticed a badly scoped variable which was probably the reason for the problem in the clean-up code, but am not able to test that as yet.

[janvanmaar]

Just a small note:
the restart is not needed if you have SSH server enabled on your Kindle in which case you can just send the following command to your Kindle instead:
ssh root@yo.ur.I.P 'dbus-send --system /default com.lab126.powerd.resuming int32:1'

[mornington]

Originally Posted by janvanmaar

Just a small note:
the restart is not needed if you have SSH server enabled on your Kindle in which case you can just send the following command to your Kindle instead:
ssh root@yo.ur.I.P 'dbus-send --system /default com.lab126.powerd.resuming int32:1'

Woah. So how can I enable SSH server on the kindle?

[janvanmaar]

usbnet hack, you can find a lot of information about it on mobileread forums (but it requires a jailbreak so you may lose your warranty, your Kindle may die etc)

[mornington]

I've hit the final revision (for me) of the script now. The newly attached version can handle include and exclude lists for authors and series and the clean up code appears to work fine as far as I can tell.

I've also added a substantial notes section to the head of the script file, detailing everything I know about how updating collections on the Kindle works.

The only remaining "bug" that I can see seems to be a bug with the Kindle, as the "Kindle Collection Manager" program I was using suffered from the same problem (i.e. that sometimes a few books are added to collections and show up in the json (and in Calibre for that matter) as having being added to those collections but the Kindle thinks they're collectionless.

If anybody has any ideas on how to solve that one, I'd like to hear them.

[mornington]

Looking at the collections.json after I manually (i.e. using the kindle's UI) put the rogue files into collections, I'm seeing some weird "ASIN" values.

To take an example, the file with the lpath:

documents\Douglas Adams\HHGTTG 01 the Hitch Hikers Guide to the Galaxy (H0tHHGttG-DAHGttGT1).mobi

Gets an ASIN (via the usual formula, and te one used in the script) of:

*ffccd1edd94d20b512450535de6a42e0181d7df3

Now, that file doesn't show up as being in a collection on the kindle's UI. However, that asin is in all the right places in the collections.json file (and Calibre thinks it's in those collections). If I then, using the kindle, put that "rogue" file into a collection and have a look at the collections.json, I see this "asin" id being used for that file:

#B000XUBC2C^EBOK

The asin from the script (*ffccd1edd94d20b512450535de6a42e0181d7df3) is still there in the collections.json but is being ignored by the kindle.

Any idea how/why the kindle is deriving this new value? It's not because of any clash of asin ids (I checked - no other files share the same asin), it's not coming from amazon.com (I have wireless switched off) and the form isn't a one-off one, as I got similar results for other "rogue" files (eg #B000R4LGMA^EBOK and #B000JMKTE6^EBOK - all of the form "#B00[XXXXXXX]^EBOK", where "[XXXXXXX]" is a sequence of 7 characters, each of which is numeric or upper-case alphabetic).

It must be being derived from the mobi files consistently in some fashion, though, and not involving timestamping (if I remove the mobi and re-send it to the kindle I get the same "rogue" id generated).

Just some food for thought: And if anybody can figure this bit out then the script can be made more reliable.

[mornington]

ok, I know I said I'd be leaving it alone, but here is the REAL final final version of the script, now that I've solved the final couple of outstanding issues:

1) If you're rebuilding all collections on your kindle with the script (the "--rebuild" command-line option) then those collections will now automatically be initially sorted into alphabetical order when viewed from the kindle's "home" page viewed by "collections". This is done by faking up the lastAccess times on the collections, and you can use the new "--sort" option to force the script to similarly "sort" all the collections on your kindle when performing an update.

2) The weird asin thing I mentioned above. This isn't "fixed" as such, but I've come up with several workarounds, which are described in the now-huge comment at the top of the script file.

Basically, I noticed that all the affected books were ones purchased from Amazon, whether they were stripped of DRM or not. So the simplest workaround is to reconvert them twice: First convert them to epub format and then convert the epubs back to mobi format. That'll strip out that embedded metadata asin and allow them to be managed as normal.

As an aside: If you use the embedded asin explicitly in the collections.json (i.e. don't do the conversion mentioned above) then Calbire can't associate that with the eBook. So presumably Calibre can't (or isn't) following that embedded-asin chain to link the books to their collections.

3) I fixed a typo in the code which was preventing creation of collection from series (I don't do this myself, which is why I didn't notice that typo earlier)


And that's DEFINITELY it: The script now does everything I wanted it to do in the first place and so, short of some workarounds to avoid having to restart the kindle and/or extract embedded asins from such files, there's nothing left for me to do to it.


[EDIT] See new thread https://www.mobileread.com/forums/sho...47#post1255247 for script file

[xmommyx]

Okay I know this is a relatively old thread...but I've read through it hoping to understand how to use this script but after several attempts I am still doing it all wrong. Could someone pretty please specify what code needs changing and where in the script I could find it to have the script create collections ONLY for AUTHORS. I don't want collections for tags, series or any other information...just authors. Thank you so much for any help and again I apologize for reopening what seems to be a dead thread.

Smile,
Crystal

[stacyk]

So - if you're using the newest scripts by mornington -
just use the command line flags --ns --nt
(No series & no tags).

If you're looking at the older code,
look for any line that has
collections.append(book['series'])