Documentation: How to Add a New Custom Column Datatype

DaltonST · 07-02-2016, 05:13 PM

Kovid,

Since I had to reverse engineer Custom Columns in order to understand what needed to be done to add a new Custom Column datatype, I documented it in the attached zip file. It includes the Calibre 2.61.0 .py files that had any changes, plus some .png files proving that it works properly.

I thought perhaps it might be useful to someone trying to do the same thing in the future.

DaltonST

kovidgoyal · 07-02-2016, 11:36 PM

The hard part is not adding a new datatype -- that would be a trivial patch of a few dozen lines. The hard part is making sure all the code that deals with metadata -- this means metadata reader/writing, export/import, catalogs, rules, metadata editor, searching, etc. all handle the new datatype.

From looking at your zip file you seem to have considered many of those. If you want your code reviewed for inclusion in calibre, please send me either a git patch or a github pull request, as described here: http://manual.calibre-ebook.com/deve...to-be-included

eschwartz · 07-03-2016, 02:50 AM

Looks like an interesting change.

Here it is in diff format.

chaley · 07-03-2016, 05:24 AM

When considering what is the "hard part", also bear in mind that adding a datatype is likely to break any third-party tools that directly read the calibre database or calibre .opf information. One example class of such tools are the alternate content servers. In addition, this kind of change is arguably like a schema change and not backwards compatible, preventing people from downgrading if required or from running both calibre V(current) and V(old) because their hardware requires it.

For this new datatype, plaintext, a much less risky approach would be to add 1 or 2 new booleans to the "display" for comments: "is_plain_text" and (possibly) "show_label". Column writers (and there aren't many of those) would need to be sure that they don't "html"ify the contents and present an appropriate editor to the user. Column readers might need to check, but given that it is currently possible that a comments column contain plain text, it isn't clear how often such a check is required. SW that copies column data (e.g., OPF stuff) probably need no changes at all.

This approach has several large advantages:

There aren't two column types that are largely the same thing. This is one reason I probably should have implemented the enumerated type using the display instead of as a distinct datatype.
Clients who read the database won't break. In rare cases they might show the data incorrectly (e.g., embedded text that looks like HTML), but that is more acceptable than falling over dead.
Clients that care about the semantics of the new booleans can adapt their behavior as required.
People who maintain third-party SW can react when they are ready, assuming a reaction is required.

kovidgoyal · 07-03-2016, 06:12 AM

I have to agree with chaley. Using extra properties in display would be much less invasive.

DaltonST · 07-03-2016, 07:33 AM

Quote:

Originally Posted by kovidgoyal

I have to agree with chaley. Using extra properties in display would be much less invasive.

Kovid,

This is what you said recently here: https://www.mobileread.com/forums/sho...d.php?t=275539

Quote:

I am not going to add labels for comments column in detail view. That would be sub-optimal for the vast majority of use cases of comments.

Since you are not going to add labels for datatype=comments in the book details view, adding a new datatype is the only other choice.

DaltonST

chaley · 07-03-2016, 07:39 AM

Quote:

Originally Posted by DaltonST

Since you are not going to add labels for datatype=comments in the book details view, adding a new datatype is the only other choice.

DaltonST

That is one reason for adding the second property I mentioned, "show_label". Such a property eliminates the need to guess the intention from the contents. If calibre can "do the right thing" with a new datatype then it can clearly do the same thing with a new property.

kovidgoyal · 07-03-2016, 12:51 PM

Quote:

Originally Posted by DaltonST

Since you are not going to add labels for datatype=comments in the book details view, adding a new datatype is the only other choice.

Not quite, chaley's proposal allows calibre to distinguish between comments fields that should be displayed as free form text and comments fields that should be displayed as single line, title like text. That means that the Book Details panel can use that information to treat these fields differently.

And his proposal has the advantage of being far less likely to break things.

07-03-2016, 05:24 AM	#4
chaley Grand Sorcerer Posts: 11,741 Karma: 6997045 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	When considering what is the "hard part", also bear in mind that adding a datatype is likely to break any third-party tools that directly read the calibre database or calibre .opf information. One example class of such tools are the alternate content servers. In addition, this kind of change is arguably like a schema change and not backwards compatible, preventing people from downgrading if required or from running both calibre V(current) and V(old) because their hardware requires it. For this new datatype, plaintext, a much less risky approach would be to add 1 or 2 new booleans to the "display" for comments: "is_plain_text" and (possibly) "show_label". Column writers (and there aren't many of those) would need to be sure that they don't "html"ify the contents and present an appropriate editor to the user. Column readers might need to check, but given that it is currently possible that a comments column contain plain text, it isn't clear how often such a check is required. SW that copies column data (e.g., OPF stuff) probably need no changes at all. This approach has several large advantages: There aren't two column types that are largely the same thing. This is one reason I probably should have implemented the enumerated type using the display instead of as a distinct datatype. Clients who read the database won't break. In rare cases they might show the data incorrectly (e.g., embedded text that looks like HTML), but that is more acceptable than falling over dead. Clients that care about the semantics of the new booleans can adapt their behavior as required. People who maintain third-party SW can react when they are ready, assuming a reaction is required.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Is there a way to add the metadata AND also the filename in a custom column?	LadyKate	Library Management	5	10-10-2014 01:13 PM
Custom column returns value based on value of another custom column?	calvin-c	Calibre	3	09-14-2013 02:24 PM
Custom yes/no column built from long text column	Philantrop	Library Management	7	03-23-2013 07:44 PM
how to move value(s) of tag column to a custom made column	zoorakhan	Library Management	0	12-08-2012 03:53 AM
Can custom book data be displayed in a custom column?	kiwidude	Development	9	03-02-2011 05:35 AM

07-02-2016, 11:36 PM	#2
kovidgoyal creator of calibre Posts: 43,856 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	The hard part is not adding a new datatype -- that would be a trivial patch of a few dozen lines. The hard part is making sure all the code that deals with metadata -- this means metadata reader/writing, export/import, catalogs, rules, metadata editor, searching, etc. all handle the new datatype. From looking at your zip file you seem to have considered many of those. If you want your code reviewed for inclusion in calibre, please send me either a git patch or a github pull request, as described here: http://manual.calibre-ebook.com/deve...to-be-included

07-03-2016, 06:12 AM	#5
kovidgoyal creator of calibre Posts: 43,856 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I have to agree with chaley. Using extra properties in display would be much less invasive.

Advert

Advert