[GUI Plugin] Kindle Collections (old)

[louhike]

Hi !
First, thanks for this plugin which seems awesome.
But I have a pro blem : I have this ugly error when I try to add the plugin :

Traceback (most recent call last):
File "/usr/lib/calibre/calibre/gui2/preferences/plugins.py", line 150, in add_plugin
add_plugin(path)
File "/usr/lib/calibre/calibre/customize/ui.py", line 370, in add_plugin
plugin = load_plugin(path_to_zip_file)
File "/usr/lib/calibre/calibre/customize/ui.py", line 76, in load_plugin
exec raw in locals
File "<string>", line 0
SyntaxError: encoding declaration in Unicode string (<string>, line 0)

Do you have any idea of what the problem may be ?

[hfpop]

Originally Posted by hfpop

For better management of my books ...

Follow up:

I have tried one more idea:
- Manual remove of internal kindle files from all collections via this plugin;
- rename of affected files
- empty documents folder
- reboot kindle
- reload documents folder
- reboot kindle
- redo the collections.json
The problem IS still there.

I have to confess, I have absolutely NO IDEA where the issue comes from.

[kiwidude]

@louhike - What version of Calibre are you using? Perhaps a very old one by the looks of it?

[meme]

Originally Posted by Cleemur

Is there a simple collections management tutorial for the kindle 3 and Calibre?
....[from other thread]:
Thanks for trying, but the link is not clear or it is not updated because for me, following the listed instructions did not sync with my experience. Calibre already comes with the kindle plug-ins. And while I am able to see book titles from my kindle in Calibre...

I see you've posted in another thread but I'll answer here. Its not exactly clear what problem you are having. If you read the first post in this thread as mentioned what exactly isn't working as expected?

[meme]

Originally Posted by hfpop

Now, my problem: many items previously on documents/ and now on documents/whatever appear as internal kindle file, and not with their author. This was not fixed even after a factory reset. As such, those files may be added to collections only on Kindle. Of course, the files added after this reshuffle are fine, but fine are as well books from the initial series.

Do you have any idea about what to do? How to reset those missing files entries?

Storing books under subdirectories under documents is perfectly fine. I use Calibre to store my books under author names on the Kindle instead of all in the top directory.

Can you see the book file on the Kindle as documents/somedirectory/somefilename ?

When using the Kindle can you see the book's title? Can you add it to a collection?

In the plugin's Edit Collection can you see the book title listed? Is it marked included in the collection you added it to on the Kindle?

Some specific examples of what you can/can't do (in the plugin/on the Kindle/etc) might help.


Kindle collections are stored on the Kindle in system/collections.json. Each book is assigned a code based on its pathname or Amazon ASIN number by the Kindle, and it is this code that is listed against the collection name in the collections.json file. If you delete a file from the Kindle then the code in the collection.json file won't refer to anything - and the plugin will list it as an Internal File (I'll be changing this to Book with no File). If you restored the file to the filesystem (if its an Amazon purchased book it can be restored anywhere since the code is in the file, but if its any other book the exact same file must be restored to the exact same path) then the code would work again and the book's title will be shown in the collection.

Unfortunately the Kindle stores some special books (dictionaries for example) somewhere else other than the documents directory. The Kindle add these special books to collections because it know their codes. The plugin can't see these special books because they are not in the documents directory. But if you add them to a collection using the Kindle, then the plugin can see their codes in the collection file - but it can only list them as 'Internal Books'.

You can just delete (or rename) the collections.json file to delete every collection - but of course you loose all your collections. Using the Edit Collection option should allow you to cleanup entries.

[louhike]

Originally Posted by kiwidude

@louhike - What version of Calibre are you using? Perhaps a very old one by the looks of it?

Oops, you're right. I was using the version on the official Ubuntu repository which is the 0.7.2... Updating it solved the problem, thanks !

[sorin]

All i want from Kindle Collection Plugin is the option to add selected books in a collection like this: select 10 books with different author, right click, a menu will appear with option Add to collection(s) which have a list of collections from my Kindle (maybe checkboxes), after i select in which collection i want to add all books i click button Eject and save.

[meme]

Where do you want to select the books from - Calibre or the Kindle?

The plugin can edit Kindle collections in 2 ways. It depends on if you use Calibre to manage your books or just use it to convert books (or haven't used it before).

The first is to automatically generate collections from Calibre using the books/tags/author names/etc you have already setup in Calibre.

To use this approach, you must select the books in Calibre and send them to your Kindle. You then need to add a Tag to each of the books you want to group together. Then run the plugin (Customize and select 'Create' for Tags, run Preview to see what will happen, then run Create if it looks ok). The plugin will then create a collection on your Kindle matching your Tag name and any other tag names you have in Calibre (and leave any other collections already on your Kindle that don't match the tag names alone). (If you have too many tags, you can customize the plugin to only Include certain tag names or to ignore certain ones).


The second way is to edit collections manually using the information stored on your Kindle (in this case Calibre is only really being used to provide a convenient way for me distribute the program, although I do use it to check some information).

To use this approach, start the plugin and select Edit. (You may need to run Customize and select OK first in the current plugin version). You now see all the books on your Kindle and all the collections that exist on the Kindle. Select the collection you want to edit. To add a new collection, enter a name in the box at the top and click the '+' button at the top. Click the checkbox in the Include column for a book to include/not-include the book in the current collection. Then click save. Then restart your Kindle and you should see the modified collections.


Using Calibre to automatically manage your collections is what the plugin was intended for. If this isn't what you want right now, then the manual edit approach also works well. And you might find as your collections grow that you will want to use Calibre more to manage your books and use the plugin to automatically manage the collections.

[sorin]

Originally Posted by meme

Where do you want to select the books from - Calibre or the Kindle?

I want to select the books from Calibre not from Kindle.
The second approach is ok but can be improved. In Edit Collection view doesn't appear Author's Column, date when document was added and document size like in the Calibre Device View.

There is an option to see in Edit Collection view all the Documents which don't belong to a collection or to sort the documents list by Collections's column value ?

My first idea was to use Calibre Device View , select some documents and here to be an option in right click menu where i can select Send to Collection(s) ..

[meme]

It seems to me that you can mimic the right click menu send to collections with the plugin. You can select the books you want, add a tag/collection name. Then run the plugin (you can put it on your right click menu), select customize, just select 'Create' for tags and put your tag/collection name in the Include column. Then run Create Collections and you will get your collection. It's essentially what you are looking for - with the benefit of reminding you what you sent to a collection on the Kindle making it easy to tag a new book and resend. OK its a couple of windows instead of one but its close.

But if you just want to create collections on the fly then you can use the Edit window. In the current plugin version you can see in the Edit collection view whether a document is in a collection or not (see the Collection column on the right), and you can select the Collection to view the books in/not-in the collection.

The next version of the plugin (currently undergoing a lot of change due to the new plugin api and to take the opportunity to cleanup the code) will add the Author and Date column to the Edit window, with sorting, and add the ability to select multiple books to add/remove.

Interestingly, if I look at my test version, it is now awfully close to the Calibre Device View. So I can imagine that I could add code to support a Send to Collections option to the plugin that does what you are thinking. It would be in addition to Edit collections but its not something I'll be looking at for some time.

[jesscat]

Sorin's post reminded me of a question I've had for a little while but have forgotten to ask. It seems to me that for an approach like that to work - to be able to right-click from device view and have an option for "send to collection" - you need to be able to see the collections in device view. Before I installed this plug-in, I remember that I used to be able to see my collections in device view, but I can't anymore. It doesn't matter so much to me because I use calibre to manage my collections, but I'm wondering why this happened - does it have something to do with the plug-in?

Also, as a reply to Sorin: my thought is that the easiest way to do what you want for right now if you want to work from the full library view when sending books to collections is to create a custom column in Calibre called "collections." Then just put in that column the names of the collections you want a book to be in, and customize the plug-in to look to that column instead of to tags. It's almost like a "send to" option - just instead of clicking "send" you type the collection name into the column. The column shows you exactly which collections the book is in, and you can sort on it, etc. That's what I do. I use collections to keep track of favorites, or unread books; that sort of thing, rather than to organize by author or series or category, so this works better for me than trying to pare down my list of tags, and it enables me to immediately see what collections a book is in just by looking at the column.

The only issue, of course, is that if you currently have many collections on your Kindle, you have to replicate those collections in your column (which, depending on how you use Calibre, might be a waste of effort if your Kindle collections are in fact based on author or tag or series, columns that already exist in Calibre) - but once you've done that, you can edit a book's collection at any time from the full library view (without changing tags or authors or anything like that - simply controlling collections) just by changing the contents of that column.

[meme]

@jesscat - A separate custom column for collections would definitely be cleaner than tags. Your post is a great explanation of why and how to do this

The plugin should not cause the collections listed against the book to disappear. The format it writes the collections out in is the same as the Kindle so it should be readable. However, if you can identify something specific there's always the possibility that something needs to be fixed.

I have noticed that the Device view won't show collections for any Amazon purchased books since it can't read the ASIN code from the file. Someday I thought of integrating into Calibre the ASIN reading code from the plugin (which was already borrowed from another collection managing program and I think based on Mobi editing software) into Calibre.

Hmmm, I think I'll put 'import collections names into column' on the wishlist...

[jesscat]

Originally Posted by meme

@jesscat - A separate custom column for collections would definitely be cleaner than tags. Your post is a great explanation of why and how to do this

Thanks! Yes, this works well for me. Actually, I had already created a column called "Collections" not too long before before you developed this plug-in - I just attempted to maintain it manually (I even posted in the forum asking Kovid if there was any way to make a column in the Calibre library view that used info from the device view, so I could at least see the collection in the library view, but there isn't). It was a pain to do that, but I really wanted to be able to see my collections in the library view even if I couldn't manage them.

So when the plug-in appeared, it was easy for me to just continue using that column. And again, I don't use collections for things like genre or author, so this is a better solution for me - I have only about six collections, and while a few of them are in fact also Calibre tags, my tag list is hundreds long, so it would be laborious to customize the plug-in using tags. Plus I wouldn't be able nearly as easily to see which collections a book is in.

Possibly you might consider adding this to the suggested custom columns in the first post. The one thing I screwed up was that when I first created the column I made it "Text" rather than "Comma separated text, like tags." When I just maintained it manually it didn't matter, but for the plug-in, to assign a book to more than one collection you need the latter (at least I assume so?). So I had to make a new column and copy the contents over.

But - I wouldn't have had to do that if there were an "import collections name into [custom] column. That is a *fabulous* idea!!

Originally Posted by meme

The plugin should not cause the collections listed against the book to disappear. The format it writes the collections out in is the same as the Kindle so it should be readable. However, if you can identify something specific there's always the possibility that something needs to be fixed.

Hmm. I don't know why this has happened then. Very few of my books are Amazon purchased, and even those I tend to de-DRM (just for my own use, but to avoid issues like the one you mention). Still, the "collections" column in my device view is now entirely empty, while it used to show my collections (that's how I manually maintained my library column before the plug-in). I don't believe the change occurred right when I first installed the plug-in (which was one of the early versions, possibly the second or third), but I don't know exactly when.

Any thoughts about how I could go about debugging this?

[meme]

Originally Posted by jesscat

Possibly you might consider adding this to the suggested custom columns in the first post.

Sounds like a good idea.

The one thing I screwed up was that when I first created the column I made it "Text" rather than "Comma separated text, like tags." When I just maintained it manually it didn't matter, but for the plug-in, to assign a book to more than one collection you need the latter (at least I assume so?). So I had to make a new column and copy the contents over.

I've just updated my test version to allow you to split a text field from Calibre into separate Collections. Its much better to use the right type of column, but adding this was fairly easy.

Still, the "collections" column in my device view is now entirely empty, while it used to show my collections (that's how I manually maintained my library column before the plug-in). I don't believe the change occurred right when I first installed the plug-in (which was one of the early versions, possibly the second or third), but I don't know exactly when.
Any thoughts about how I could go about debugging this?

There could be a minor difference in the way the file format is handled between Calibre, Kindle, and the plug-in. I'll have to do some testing with clean collections files and look at Calibre's code for handling the collections.



@jay2222 - a long time ago you asked about using tags like tgSKI in your titles for items send from Read It Later. Well, with the addition of the title to the customization dialog and the ability to split on a character in the next version of the plugin, you'll be able to create collections for your tags.

[pchrist7]

Originally Posted by jesscat

Hmm. I don't know why this has happened then. Very few of my books are Amazon purchased, and even those I tend to de-DRM (just for my own use, but to avoid issues like the one you mention). Still, the "collections" column in my device view is now entirely empty, while it used to show my collections (that's how I manually maintained my library column before the plug-in). I don't believe the change occurred right when I first installed the plug-in (which was one of the early versions, possibly the second or third), but I don't know exactly when.

Any thoughts about how I could go about debugging this?

Hi jesscat. Noticed the same some months ago. Didn't know for sure whether the change was due to Calibre updates, or K3 firmware update. Now Calibre shows no collection info in device view - "some months ago" some of my collections were shown. Never figured it out.
Importing collection into custom column, uhmmmm... I like it