Creating Kindle Collections from Calibre's Data

[ch4os]

Thanks for catching my eye on azw files.

I've changed 466 line from

print collDesc.encode('utf-8')

to

print collDesc

and it worked, so i don't know if u need to convert all strings to utf-8 before print, i think calibre ensure proper encoding of metadata file.

edit.
After while i think it could be os depedent, i'm running your script at linux where utf-8 is default encoding, windows (with kernel 6.x - vista, 7, 2k8) use utf-16. Calibre force utf-8 to write metadata file? or use system encoding?

[mornington]

Originally Posted by ch4os

i don't know if u need to convert all strings to utf-8 before print, i think calibre ensure proper encoding of metadata file.

edit.
After while i think it could be os depedent, i'm running your script at linux where utf-8 is default encoding, windows (with kernel 6.x - vista, 7, 2k8) use utf-16. Calibre force utf-8 to write metadata file? or use system encoding?

I'm not sure myself, but I do know that if I didn't explicitly encode everything from Calibre to utf-8 then the script just constantly fell over on the slightest deviation from plain-ASCII characters. There's probably a more robust way to handle this kind of thing in Python, but like I said I don't know Python...

Glad the edit works for you, though. Did you try out the most recent version of the script, incidentally? I'm still not able to test it on the kindle, so I'd appreciate some reassurance that it works.

[ch4os]

I've tested latest version of your script and with my 'edit' everything works fine, great job it should be built-in into calibre

[beckywc]

I have a custom column thats called "In Collection" this column is a text, but with a set of permitted values. These values are the collections I have currently on my Kindle. When I add a book to my library I select which collection it will go in.

How would I be able to alter you script to fit my set up?

Thanks

[mornington]

Originally Posted by beckywc

I have a custom column thats called "In Collection" this column is a text, but with a set of permitted values. These values are the collections I have currently on my Kindle. When I add a book to my library I select which collection it will go in.

How would I be able to alter you script to fit my set up?

Thanks

Pretty much the same as for tags/authors/series, except that you'd be retrieving the values from a custom column instead. Those values are also stored in calibre.metadata but you may need to dig a little deeper into the json structure to extract them.

You can open your calibre.metadata in a text editor to see exactly how its structured (and so from where you need to extract the values you want): It's just a json format structure.

Probably it'd be something like this to obtain the values to parse (based on a quick glance at the json structure just now - I've not tested this at all):

mycustom = book["user_metadata"]["#mycustomcolumnid"]["#value#"]

So, parse the calibre data in parseCalibreMetadata() then use your parsed data to build your collections in createCollectionsFromCalibre(). Follow through how this is done for tags and you should be able to handle your custom column similarly.

[beckywc]

Wow sounds complicated, but will give it a shot. Thanks

[beckywc]

Please bare with me, I've never done this before and so I'm trying to get an understanding here. I checked the calibre.metadata and found this pertaining to my custom column:

Spoiler:

"languages": [],
"user_metadata": {
"#collection": {
"is_category": true,
"#extra#": null,
"kind": "field",
"is_custom": true,
"name": "In Collection",
"datatype": "enumeration",
"rec_index": 21,
"search_terms": [
"#collection"
],
"link_column": "value",
"label": "collection",
"colnum": 1,
"column": "value",
"#value#": "Various Authors G - M",
"category_sort": "value",
"table": "custom_column_1",
"is_editable": true,
"is_multiple": null,
"display": {
"enum_values": [
"Author's Series A - M",
"Author's Series N - Z",
"Various Authors A - F",
"Various Authors G - M",
"Various Authors N - Z",
"Various Authors A - F Read",
"Various Authors G - M Read",
"Various Authors N - Z Read",
"Reference",
"Christian",
"Self-Help",
"Short Stories",
"Author: Alexandria Ivy",
"Author: Brenda Jackson",
"Author: Bridget Midway",
"Author: Camille Anthony",
"Author: Cherise Sinclair",
"Author: Christina Dodd",
"Author: Christine Feehan",
"Author: D. McEntire",
"Author: Dana Marie Bell",
"Author: Diana Palmer",
"Author: Eve Vaughn",
"Author: Gena Showalter",
"Author: J.C. Wilder",
"Author: Jaden Sinclair",
"Author: Jaid Black",
"Author: J. R. Ward",
"Author: Kresley Cole",
"Author: Lara Adrian",
"Author: Lauren Dane",
"Author: Leah Brooke",
"Author: Lena Mathews",
"Author: Linda Lael Miller",
"Author: Lora Leigh",
"Author: Lorelei James",
"Author: Lori Foster",
"Author: Madison Layle",
"Author: Marie Rochelle",
"Author: Marilyn Lee",
"Author: Marisa Chenery",
"Author: Maya Banks",
"Author: Melissa Schroeder",
"Author: Mima",
"Author: Moira Rogers",
"Author: Nalini Singh",
"Author: Rae Morgan",
"Author: Rhyannon Byrd",
"Author: Sarah McCarty",
"Author: Shelly Laurenston",
"Author: Sherrilyn Kenyon",
"Author: Sherryl Woods",
"Author: Shiloh Walker",
"Author: Stephanie Morris",
"Author: Tressie Lockwood",
"Author: Vivi Andrews",
"Author: Vivian Arend",
"Author: Wendy Stone",
"Author: Zena Wynn"
]
}

What do I do now with the script?

[mornington]

Originally Posted by beckywc

Wow sounds complicated, but will give it a shot. Thanks

That's why I didn't do it that way in the script, but used tags/authors/series instead. Much easier.

If you're not comfortable rewriting the script for your specific needs, why not just copy the custom column values into your tags using Calibre's bulk metadata edit tool?

To do this: Select books to edit in calibre, right click, Edit metadata in bulk, "Search and Replace" tab. Set Search mode to "Regular Expression", Search field to your column name, search for to "(.+)" (without the quotes), replace with to "-\1" (again, without the quotes), destination field to "tags", mode to "append to field", ensure "use comma" is ticked then apply

Back up your library before doing this, but it should create a new tag for each book and give that tag a dash prefix so that the script will detect it and turn it into a collection. If you don't want the collection names to have a prefix then change prefixTag in the script to an empty string (the script uses the dash to identify tags to turn into collections and then removes the dash before adding the prefixTag to the name).

[Mixx]

Dear mornington,

after much prep work in my calibre data, I was ready to start with revamping the collections on my Kindle today.

Alas, I am getting this error message:

ERROR loadCalibre: 'ascii' codec can't encode character u'\xe4' in position 23: ordinal not in range(128)

It apparently has to do with character sets. My Kindle collections or the Calibre tags do not have non-ascii, but my author names do.

What can I do? Thanks for any help.

Regards, Mixx

[mornington]

Originally Posted by Mixx

Dear mornington,

after much prep work in my calibre data, I was ready to start with revamping the collections on my Kindle today.

Alas, I am getting this error message:


It apparently has to do with character sets. My Kindle collections or the Calibre tags do not have non-ascii, but my author names do.

What can I do? Thanks for any help.

Regards, Mixx

If you're using the latest version of the script, I've no idea what could be causing that error, sorry.

If you're using an old version then try using the latest one.

[Mixx]

Yeah, I am using the latest version of the script. Is there a particular version of Python that needs to be used, perhaps?

Thanks, Mixx

[MrTissues]

I am getting an odd syntax error with my script, I have no idea what is wrong. I am not too familiar with Python. Could someone maybe take a look? It is on line 54.

Thanks for any help.

[Perkin]

The print in python3 needs braces () for the arguments.
(I'm only just starting out in python, noticing differences between 2.7 and 3.#)

[MrTissues]

So instead if the braces I should use the parentheses?

[chaley]

My guess is that you should use python 2.6 or 2.7 instead of 3. There are *many* differences between python 2 and python 3, and you will have to fight your way through all of them.