Can;t modify user columns with set_metadata?

Silek · 07-08-2011, 09:49 PM

Hello.

I am working on a script to update metadata in Calibre 0.8.8.

I have a perl script using XML::Simple with JSON to decode, and then encode the user_metadata->content fields.

When I run calibredb set_metadata on the OPF file I generate, I do not get an error, but I also do not get the user columns set. What is strange to me also is when I add fields to the user columns in the GUI, then run show_metadata, I do not see the entries in the OPF file for the user column.

Am I doing something wrong here? Does set_metadata not work for user/custom column data? Is show_metadata not supposed to show entries in these fields?

chaley · 07-09-2011, 04:21 AM

Quote:

Originally Posted by Silek

When I run calibredb set_metadata on the OPF file I generate, I do not get an error, but I also do not get the user columns set.

Command line set_metadata will update custom columns.

The format for custom columns in the OPF is complicated. My guess is that you have it wrong and the data is being ignored. Change a book using the GUI to have the data you expect when you run set_metadata, then look at the metadata.opf file for the book. Ensure that your script generates exactly the same stuff.

Quote:

What is strange to me also is when I add fields to the user columns in the GUI, then run show_metadata, I do not see the entries in the OPF file for the user column.

I see them in both the OPF and columnar output. For example, dumping the OPF for one of my books in my test library I get

Spoiler:

However, during testing I did find a bug in the OPF renderer that has been there for over a year (!). The generated OPF has empty values for tags-like custom columns. If you are testing exclusively with an is_multiple datatype, that would explain you not seeing. Non-multiple columns do render values correctly, as shown above. Note that the columnar output does not suffer from this problem.

Silek · 07-09-2011, 09:15 AM

Thanks for the reply.

The rendering problem you mention may be why I am not seeing the GUI entries. I am using the "multiple text, show in tag browser" column type.

What my script is doing is (with perl XML:Simple and JSON) slurping a template.opf from an empty book, and in the case of custom columns, using JSON to decode the string, alter the resultant perl structure, and then encode and put the changed string back in the XML.

The XML format is somewhat different, but looks similar enough. I have even taken the "content" field out and cut and pasted it into the template and that also has not worked.

Now the order of the JSON fields is different, but I thought that would not matter.

The template entry for a custom field is:

<meta name="calibre:user_metadata:#characters" content="{"is_category": true, "#extra#": null, "kind": "field", "is_custom": true, "is_csp": false, "colnum": 16, "column": "value", "rec_index": 25, "search_terms": ["#characters"], "link_column": "value", "is_multiple2": {"cache_to_list": "|", "list_to_ui": " & ", "ui_to_list": "&"}, "is_multiple": "|", "datatype": "text", "#value#": [], "category_sort": "value", "table": "custom_column_16", "is_editable": true, "label": "characters", "display": {"is_names": true}, "name": "Characters"}"/>

My entry is:

<meta>
<name>calibre:user_metadata:#characters</name>
<content>{"link_column":"value&quot ;,"is_csp":false,"table":&quot ;custom_column_16","search_terms":["#characters"],"rec_index":25,"display":{&qu ot;is_names":true},"is_custom":true ,"is_category":true,"kind":&qu ot;field","#value#":"[ Sasha Hammer, Iron Man, War Machine, Dum Dum Dugan, Pepper Potts, Maria Hill, Ezekiel Stane ]","colnum":16,"is_editable&quo t;:true,"category_sort":"value&quot ;,"column":"value","name& quot;:"Characters","is_multiple2&qu ot;:{"ui_to_list":"&",&quo t;cache_to_list":"|","list_to_ ui":" & "},"#extra#":null,"label" :"characters","is_multiple":&q uot;|","datatype":"text"} </content>
</meta>

Due to the rendering problem, I do not have a good idea of what the entry is supposed to look like. I figure the #value# entry is wrong, but I am not getting an error from calibredb set_metadata. Since I did not get an error, I did not know to dig in to the format (or even, where in the format to look).

Any pointers on what is wrong with the JSON above, or where to look for either helpful error messages or a sample of a working XML snippet would be greatly appreciated.

I am using this data type because I wanted these entries to show in the tag browser. I suppose I could use dc:subject tags and add then specifically as tags, and leave them as one long comma delimited string in a text field, but that feels kludgy.

Thanks again.

Lawrence

chaley · 07-09-2011, 09:36 AM

Quote:

Originally Posted by Silek

The rendering problem you mention may be why I am not seeing the GUI entries. I am using the "multiple text, show in tag browser" column type.

Look at the metadata.opf file in the books folder in the library. They do not exhibit the problem.

Quote:

Now the order of the JSON fields is different, but I thought that would not matter.

It won't.

Quote:

The template entry for a custom field is:

My entry is:

<meta>
<name>calibre:user_metadata:#characters</name>
<content>

I have no idea if calibre's xml engine will consider this alternate structure as equivalent to its own.

Quote:

Due to the rendering problem, I do not have a good idea of what the entry is supposed to look like.

As mentioned above, you can see calibre's version in the metadata.opf file.

Quote:

I figure the #value# entry is wrong, but I am not getting an error from calibredb set_metadata. Since I did not get an error, I did not know to dig in to the format (or even, where in the format to look).

The question is whether or not the JSON decode will work. The decoder might not throw an error, but could come up with different values. For example, I note that your string constants in #value# are not quoted. JSON will probably ignore them.

Quote:

I am using this data type because I wanted these entries to show in the tag browser. I suppose I could use dc:subject tags and add then specifically as tags, and leave them as one long comma delimited string in a text field, but that feels kludgy.

There are a lot of types that show up in the tag browser, such as text, enumeration, text/multiple, composite w/show in tag browser checked. You should use the one that makes sense for your data.

I also note that your template declares that the multiple text entry contains names (the is_names in the display attribute). Why did you check that box for tags-like items?

Silek · 07-09-2011, 02:40 PM

Thanks. In the book directory I see the metadata.opf.

I checked for "people's names" because usually they are people's names. That said, I did not actually check the documentation as to what the box did.

I am going to get the right format into the JSON and see if that resolved the problem.

Thank you very much for your quick replies. I did not even think to check the book directory metadata.opf.

07-08-2011, 09:49 PM	#1
Silek Junior Member Posts: 3 Karma: 10 Join Date: Jul 2011 Device: color nook	Can;t modify user columns with set_metadata? Hello. I am working on a script to update metadata in Calibre 0.8.8. I have a perl script using XML::Simple with JSON to decode, and then encode the user_metadata->content fields. When I run calibredb set_metadata on the OPF file I generate, I do not get an error, but I also do not get the user columns set. What is strange to me also is when I add fields to the user columns in the GUI, then run show_metadata, I do not see the entries in the OPF file for the user column. Am I doing something wrong here? Does set_metadata not work for user/custom column data? Is show_metadata not supposed to show entries in these fields?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
set_metadata and covers	DarkAbsynthe	Calibre	12	09-19-2020 02:38 PM
User PNG's for User Columns	transmitthis	Library Management	7	06-29-2011 04:45 PM
Can I modify the dictionary?	youngsmoke	Fictionwise eBookwise	0	11-30-2010 04:48 PM
User Defined Columns	jjansen	Calibre	3	03-17-2010 05:33 PM
Command line calibredb set_metadata problem	Nedan	Calibre	1	02-27-2009 08:44 PM

07-09-2011, 09:15 AM	#3
Silek Junior Member Posts: 3 Karma: 10 Join Date: Jul 2011 Device: color nook	Thanks for the reply. The rendering problem you mention may be why I am not seeing the GUI entries. I am using the "multiple text, show in tag browser" column type. What my script is doing is (with perl XML:Simple and JSON) slurping a template.opf from an empty book, and in the case of custom columns, using JSON to decode the string, alter the resultant perl structure, and then encode and put the changed string back in the XML. The XML format is somewhat different, but looks similar enough. I have even taken the "content" field out and cut and pasted it into the template and that also has not worked. Now the order of the JSON fields is different, but I thought that would not matter. The template entry for a custom field is: <meta name="calibre:user_metadata:#characters" content="{"is_category": true, "#extra#": null, "kind": "field", "is_custom": true, "is_csp": false, "colnum": 16, "column": "value", "rec_index": 25, "search_terms": ["#characters"], "link_column": "value", "is_multiple2": {"cache_to_list": "\|", "list_to_ui": " & ", "ui_to_list": "&"}, "is_multiple": "\|", "datatype": "text", "#value#": [], "category_sort": "value", "table": "custom_column_16", "is_editable": true, "label": "characters", "display": {"is_names": true}, "name": "Characters"}"/> My entry is: <meta> <name>calibre:user_metadata:#characters</name> <content>{"link_column":"value&quot ;,"is_csp":false,"table":&quot ;custom_column_16","search_terms":["#characters"],"rec_index":25,"display":{&qu ot;is_names":true},"is_custom":true ,"is_category":true,"kind":&qu ot;field","#value#":"[ Sasha Hammer, Iron Man, War Machine, Dum Dum Dugan, Pepper Potts, Maria Hill, Ezekiel Stane ]","colnum":16,"is_editable&quo t;:true,"category_sort":"value&quot ;,"column":"value","name& quot;:"Characters","is_multiple2&qu ot;:{"ui_to_list":"&",&quo t;cache_to_list":"\|","list_to_ ui":" & "},"#extra#":null,"label" :"characters","is_multiple":&q uot;\|","datatype":"text"} </content> </meta> Due to the rendering problem, I do not have a good idea of what the entry is supposed to look like. I figure the #value# entry is wrong, but I am not getting an error from calibredb set_metadata. Since I did not get an error, I did not know to dig in to the format (or even, where in the format to look). Any pointers on what is wrong with the JSON above, or where to look for either helpful error messages or a sample of a working XML snippet would be greatly appreciated. I am using this data type because I wanted these entries to show in the tag browser. I suppose I could use dc:subject tags and add then specifically as tags, and leave them as one long comma delimited string in a text field, but that feels kludgy. Thanks again. Lawrence

07-09-2011, 02:40 PM	#5
Silek Junior Member Posts: 3 Karma: 10 Join Date: Jul 2011 Device: color nook	Thanks. In the book directory I see the metadata.opf. I checked for "people's names" because usually they are people's names. That said, I did not actually check the documentation as to what the box did. I am going to get the right format into the JSON and see if that resolved the problem. Thank you very much for your quick replies. I did not even think to check the book directory metadata.opf.

Advert

Advert