need custom column built from other cols, behave like tags but with author-like separ

[masp]

I've created a very useful custom column (built from other columns) that hold canonicalized author names: it works great (I used author links to hold the canonical form of any given name, and then this regex: {:'list_re(author_links(":", " & "), " & ", ".*?([^:]+):?$", "\1")'}). All is good, except that I had to use the "built form other columns" type rather than the "built from other columns, behaves like tags" type because the names may contain commas, which causes havoc with the tag-like behaviour. But as a result, when I look at my new category in the tag browser, it doesn't behave like a list: for multiple authors I see single entries such as "bob & jane & clare". So what I believe I need (but can't find) is a way to create a custom column that is "built from other columns" and "behaves like authors". I do see that if I create a "comma-separated text, like tags" custom column then I can set "contains names", which seems to do what I want (ie. make the separator & rather than ",") but is it possible to get that behaviour from the "built from other columns, behaves like tags" type? (ie. ampersands rather than commas as separator).

Thanks!

Mike

[masp]

To be more clear: does anyone know if it's possibly to create a composite column that uses '&' rather than ',' as its list separator?

[BetterRed]

Your thread shows its been viewed 70+ times. So it's probably safe to assume none of those viewers know if it's possible to do what you want.

If you state why you want a such a composite column, i.e. what problem are you trying to solve, you might get some answers.

BR

[chaley]

No, it isn't possible. Composite columns always use a comma as a separator. That won't change, especially because the "&" implies item order as well as the separator.

[ilovejedd]

Just a thought. If the contents of the composite column doesn't change frequently, perhaps instead of a composite column, you can instead create a custom authors column and use bulk search & replace to copy the template result to the custom authors column.

[masp]

Thanks for the replies:

@chaley: thanks for the definitive answer: I figured it was not possible, but I just wanted to rule it out before I gave up on that solution.

@BetterRed: the reason is that I've created a composite column that is a mapping of author names to canonical author names (via a lookup table that's implemented as the "author link" column under "manage authors"). That part works fine: the new column holds all the author names, and they've all been canonicalized where necessary. But I then wanted to be able to view those (canonical) authors under the tag browser: however, if I retain "&" as a separator (which is what I was hoping I could do) then the individual names are lumped together as a simple string (eg. "bob & jane & clare" is treated as if it was a single author's name) and if I use "," as a separator then the names are split out individually as I want, but also any additional commas cause extra (bogus) splitting (eg. "Lloyd Biggle, Jr." becomes two authors: "Lloyd Biggle" and "Jr."). So I'm at a loss as to how to achieve the tag-like behaviour (canonical author names being split out in the tag browser) without having to eliminate the use of commas in my authors column (which seems like an ugly workaround). I was hoping that I could select something like "contains names" and have the composite column output treated as an ampersand-separated list, but that is apparently not possible. Any other solutions to this problem are welcome, naturally

@ilovejedd: thanks for the suggestion: it would be a good workaround, but the column does change fairly often: whenever I add a new book or article, or whenever I clean up the authors on existing books (which happens fairly often still because I didn't take a lot of care when I started adding books!).

[ilovejedd]

Originally Posted by masp

@ilovejedd: thanks for the suggestion: it would be a good workaround, but the column does change fairly often: whenever I add a new book or article, or whenever I clean up the authors on existing books (which happens fairly often still because I didn't take a lot of care when I started adding books!).

Hmm, I seem to recall there might be a plugin that copies the contents of a composite column to custom column when you quit Calibre or something if the values have changed. You might want to look into that.

[masp]

Thanks: I'll look into it. It is a shame that composite columns can't share the "contains names" feature along with the "comma-separated text like tags" columns: that would be perfect! I wonder if it's a code limitation or a philosophical limitation?

[chaley]

Originally Posted by ilovejedd

Hmm, I seem to recall there might be a plugin that copies the contents of a composite column to custom column when you quit Calibre or something if the values have changed. You might want to look into that.

You would use a "Library Closed" plugin, which would have full access to calibre's database. If I were doing it I wouldn't bother with the composite column, instead calculating the value in the plugin then storing that value into an "contains names" column.

Two examples of library closed plugins: Save Composite Custom Columns and Save Virtual Libraries To Column

Originally Posted by masp

Thanks: I'll look into it. It is a shame that composite columns can't share the "contains names" feature along with the "comma-separated text like tags" columns: that would be perfect! I wonder if it's a code limitation or a philosophical limitation?

Both.

Both the Author columns and "Contains names" columns enforce ordering, ensuring that the authors always appear in the order they were originally listed. When custom columns were developed, comma-separated lists did not enforce order; a fetched list could have a different order from the originally stored list. As of calibre 3, all multiple fields enforce order, but no changes were made (see next issue).

More to the philosophical side: compatibility. Adding & as a separator would break anything that processes the list as text or constructs text from the list. For me, the benefit does not justify the pain.

[ilovejedd]

Originally Posted by chaley

You would use a "Library Closed" plugin, which would have full access to calibre's database. If I were doing it I wouldn't bother with the composite column, instead calculating the value in the plugin then storing that value into an "contains names" column.

Two examples of library closed plugins: Save Composite Custom Columns and Save Virtual Libraries To Column

Interesting. I don't think I've used either. Which plugin should one install to calculate the values of a template and save that to a custom column? Or would both of them work?

[chaley]

Originally Posted by ilovejedd

Interesting. I don't think I've used either. Which plugin should one install to calculate the values of a template and save that to a custom column? Or would both of them work?

Neither of the plugins I referenced would do that. You (or someone) would need to write a new plugin that does something like the following:

For each book in the library:

1. Evaluates a template supplied in the plugin parameters against the book.
2. Compares the resulting string against the contents of the custom column specified in the parameters.
3. If different, update the custom column with the new data.

Tomorrow I will take a look at how much work it is. I suspect that there are easy cases, but that there will be problems when the source and destination aren't identical types.

[masp]

Thanks @chaley: out of curiosity, how does the parsing work with the "comma-separated text like tags" column when "contains names" is selected? That does get parsed as a list (as indeed does the authors column), despite (or indeed due to) having ampersand as separator: ie. if I enter "bob & jane" then I see "bob" and "jane" as separate entities in the tag browser. I had hoped that because that column type could be split on "&" (with the "contains names" option selected) then so could the "behaves like tags" composite column (if that same option was available in that context).

[davidfor]

How about changing the comma in the names to something else? Ideally, a character that looks like a comma, but, calibre won't treat as a comma. I'm to lazy to work out the best character, so I've done this with an "@"

{:'re(list_re(re(author_links(":", " & "),",","@"), " & ", ".*?([^:]+):?$", "\1"),"&",",")'}

That gets pretty close to what I think you want.

[masp]

Yes that's a good idea: some unicode character that indistinguishable from a comma. Assuming "contains names" is not going to become available for composite columns, that's the best suggestion (for my use case) I've seen! Thanks!

[chaley]

Originally Posted by masp

Thanks @chaley: out of curiosity, how does the parsing work with the "comma-separated text like tags" column when "contains names" is selected? That does get parsed as a list (as indeed does the authors column), despite (or indeed due to) having ampersand as separator: ie. if I enter "bob & jane" then I see "bob" and "jane" as separate entities in the tag browser. I had hoped that because that column type could be split on "&" (with the "contains names" option selected) then so could the "behaves like tags" composite column (if that same option was available in that context).

Non-composite is-multiple columns are stored in the database as lists. When moving to/from text form, either the separator character (comma or ampersand) is added or the text is split on the separator character. It is rather rare in calibre that the text form is needed internally. Places where the list is processed can get the list directly without any parsing. The separator character is specified in the column description, and has been since custom columns were introduced.

Composite columns are not stored at all, but are computed on demand. If a list is needed then the template result is split at the separator, hard-coded as a comma. Introducing a variable separator would introduce the requirement that all list use sites to check the column description or they could break. I have no idea how many external tools, plugins, or the like would break, but I am sure it is a lot of them.