Rules, templates, and functions for column coloring and composed icons

[readin]

I want to visually format my library (the column grid) as follows:

1) Color all columns red if the book contains no ebook formats (although it might have other formats, such as LNK or MP3).

2) Fill a column with "composed icons" (a series of icons) to represent different statuses of the book:
a) if I own a physical copy of the book (#own:"=p")
b) if the book is part of a series
c) if the book has a LNK format (a folder shortcut)

There seem to be many different ways to accomplish the above. Following are some of the things I've tried. Some work, and I'm requesting help to fix the things that don't.

Ideally I would like to use the most efficient method possible, to minimize any impact on calibre's performance. So, to the best of my knowedge, the methods I describe below are arranged in order of increasing efficiency (but please correct me if I'm wrong )

METHOD 1 - Basic rules - THIS WORKS

1) Color all columns red if the book contains no ebook formats:

Preferences > Look and Feel > Column coloring:

Set the color of <All Columns> to <red>
if the <Formats> column <does not have> values <EPUB, MOBI, PDF>

(Note: Since a book might have other formats such as LNK or MP3, I can't just use the rule: if the <Formats> column <is not set>.)

2) Fill a column with "composed icons":

Preferences > Look and Feel > Column icons:

Set the <composed icons w/no text> of <Icons> to <book.png>
if the <Own> column <has> value <p>

Set the <composed icons w/no text> of <Icons> to <series.png>
if the <Series> column <is set>

Set the <composed icons w/no text> of <Icons> to <folder.png>
if the <Formats> column <has> value <LNK>


METHOD 2 - Multiple advanced rules for column icons - THIS WORKS

Preferences > Look and Feel > Column icons:

Advanced Rule: Set <composed icons w/no text> for column <Icons>:
program:
contains(field('#own'), 'p', 'book.png', '')

Advanced Rule: Set <composed icons w/no text> for column <Icons>:
program:
test(field('series'), 'series.png', '')

Advanced Rule: Set <composed icons w/no text> for column <Icons>:
program:
contains(approximate_formats(), 'LNK', 'folder.png', '')


METHOD 3 - Single advanced rule for column icons - PARTIALLY WORKS

Advanced Rule: Set <composed icons w/no text> for column <Icons>:

program:
	list_union(
		'',
		strcat(
			contains(field('#own'), 'p', 'book.png,', ''),
			test(field('series'), 'series.png', ''),
			contains(approximate_formats(), 'LNK', 'folder.png,', ''),
		), ',')

This rule sort of works: an icon is displayed if only one of the tests is true, but if two or more tests are true, no icons are displayed.

Note: I got the idea for the above rule from chaley's posts here and here.


METHOD 4 - User-defined template functions

As chaley has suggested (e.g. here), a custom template function would probably be the most efficient method, but I haven't been able to get very far.

For example:

1) Color all columns red if the book contains no ebook formats:

Preferences > Look and Feel > Template Functions:

name: has_ebook
arg count: 1
doc: has_ebook(val) -- evaluate whether 'val' (passed as 'approximate_formats()') has any of the given ebook formats, and return "true" or "false"
program code:

def evaluate(self, formatter, kwargs, mi, locals, val):
	contains(val, 'EPUB|MOBI|PDF', 'true', 'false')

Preferences > Add your own columns > Add custom column:

Column type: Column built from other columns
Template (press F2 in grid to edit):

program:
	f = approximate_formats();
	has_ebook(f)

Result:
EXCEPTION: global name 'contains' is not defined


If you've made it this far, thanks for reading! It's been an interesting exploration, and I'd be very grateful for any feedback.

calibre 2.63 [64bit] on Windows 10 Pro

[chaley]

Originally Posted by readin

METHOD 3 - Single advanced rule for column icons - PARTIALLY WORKS

Advanced Rule: Set <composed icons w/no text> for column <Icons>:

program:
	list_union(
		'',
		strcat(
			contains(field('#own'), 'p', 'book.png,', ''),
			test(field('series'), 'series.png', ''),
			contains(approximate_formats(), 'LNK', 'folder.png,', ''),
		), ',')

This rule sort of works: an icon is displayed if only one of the tests is true, but if two or more tests are true, no icons are displayed.

There are two errors in the template, one because I made a mistake in my suggestions and the other just because.

1) The template must return a colon-separated list, not a comma-separated list.
2) 'series.png' does not contain a separator.

The following template, adapted from yours to use a column and icons I have available, works.

program:
	list_union(
		'',
		strcat(
			contains(field('#enum'), 'two', 'add_book.png:', ''),
			test(field('series'), 'arrow-up.png:', ''),
			contains(approximate_formats(), 'epub', 'debug.png:', ''),
		), ':')

========================================

METHOD 4 - User-defined template functions

As chaley has suggested (e.g. here), a custom template function would probably be the most efficient method, but I haven't been able to get very far.

For example:

1) Color all columns red if the book contains no ebook formats:

Preferences > Look and Feel > Template Functions:

name: has_ebook
arg count: 1
doc: has_ebook(val) -- evaluate whether 'val' (passed as 'approximate_formats()') has any of the given ebook formats, and return "true" or "false"
program code:

def evaluate(self, formatter, kwargs, mi, locals, val):
	contains(val, 'EPUB|MOBI|PDF', 'true', 'false')

Custom template functions are written in python, not in the template language. Python for a template function that does what I think you are asking for is

def evaluate(self, formatter, kwargs, mi, locals):
	import re
	fmt_data = mi._proxy_metadata.db_approx_formats
	fmt_data = ','.join(v.upper() for v in fmt_data)
	return '' if re.search('EPUB|MOBI|PDF', fmt_data) is None else 'Yes'

You can get the python equivalents for built-in formatter functions by looking at their source code in the template editor or on this page.

You would call it like this:

program:
	has_ebook()

Using the template tester to debug the function(s) is faster than changing the template in a custom column. I have added it (the tester) to the right-click context menu for the main library.

[readin]

Originally Posted by chaley

METHOD 3 - Single advanced rule for column icons ... The following template, adapted from yours to use a column and icons I have available, works...

Thank you chaley, that works beautifully.

I modified it to display blank icons if a match is not found. This aligns the icons so that they are always displayed in the same position.

program:
	list_union(
		'',
		strcat(
			contains(field('#own'), 'p', 'book.png:', 'blank.png:'),
			test(field('series'), 'series.png:', 'blank.png:'),
			contains(approximate_formats(), 'LNK', 'folder.png:', 'blank.png:'),
		), ':')

I've attached a screenshot of the result below.

Using the template tester to debug the function(s) is faster than changing the template in a custom column. I have added it (the tester) to the right-click context menu for the main library.

Nice! I gave it a keyboard shortcut.

METHOD 4 - User-defined template functions ... a template function that does what I think you are asking for is...

Thanks very much!

I am now using the has_ebook() function you provided as follows:

Preferences > Look and Feel > Column coloring:

Advanced Rule for <All Columns>:

program:
	test(has_ebook(), '', 'red')

Using these two methods -- a template for composed icons, and a function for coloring books red -- is probably much more efficient than using a bunch of basic rules, especially because I will be adding a few more icons and ebook formats to them.

Even though I've dabbled in programming over the years, I don't know Python, and there's a lot in your function that I don't understand. Creating a function instead of a template for the composed icons may be beyond me.

So thanks for helping me to get this far.

[chaley]

Originally Posted by readin

Thank you chaley, that works beautifully.

You are welcome. It is fun to work with people who do their homework.

I am now using the has_ebook() function you provided as follows:

Preferences > Look and Feel > Column coloring:

Advanced Rule for <All Columns>:

program:
	test(has_ebook(), '', 'red')

You should combine the test into the function, assuming that you don't have some other use for it. Something like this should work.

def evaluate(self, formatter, kwargs, mi, locals, color):
	import re
	fmt_data = mi._proxy_metadata.db_approx_formats
	fmt_data = ','.join(v.upper() for v in fmt_data)
	return '' if re.search('EPUB|MOBI|PDF', fmt_data) is None else color

Be sure to set the number of parameters to 1.

Given that it returns 'color' without change, you could use the same function for an icon simply by passing 'foo.png' instead of 'red'.

Using these two methods -- a template for composed icons, and a function for coloring books red -- is probably much more efficient than using a bunch of basic rules, especially because I will be adding a few more icons and ebook formats to them.

Yes, that is certain. Combining basic rules into one advanced rule is a win simply starting to evaluate a rule costs something. If you can avoid complexity so much the better.

[readin]

Originally Posted by chaley

You are welcome. It is fun to work with people who do their homework.

You should combine the test into the function ...

Yes! ( ) Done.

Given that it returns 'color' without change, you could use the same function for an icon simply by passing 'foo.png' instead of 'red'.

Oooh, that got me cooking! Here's what I've got so far:

program:
	strcat(
		icon_if_set('series', 'series'),
		':',
		icon_if_fmt('LNK', 'folder'),
	)

name: icon_if_set
arg count: 2
doc: icon_if_set(field, icon) -- if 'field' value is set, returns icon filename ('icon' + '.png:'), else returns 'blank.png'
program code:
def evaluate(self, formatter, kwargs, mi, locals, field, icon):
	if mi.get(field):
		return icon + '.png'
	return 'blank.png'

name: icon_if_fmt
arg count: 2
doc: icon_if_fmt(format, icon) -- if format is found, returns icon filename ('icon' + '.png'), else returns 'blank.png'
program code:
def evaluate(self, formatter, kwargs, mi, locals, format, icon):
	import re
	fmt_data = mi._proxy_metadata.db_approx_formats
	fmt_data = ','.join(v.upper() for v in fmt_data)
	if re.search(format, fmt_data):
		return icon + '.png'
	return 'blank.png'

But I'm stuck on this one:

name: icon_if_val
arg count: 3
doc: icon_if_val(field, val, icon) -- if 'field' contains 'val', returns icon filename ('icon' + '.png'), else returns 'blank.png'
program code:
def evaluate(self, formatter, kwargs, mi, locals, field, val, icon):
	import re
	if re.search(val, mi.get(field)):
		return icon + '.png'
	return 'blank.png'

Result:
EXCEPTION: expected string or buffer

[chaley]

Thinking about this a bit more, I think that the best solution is

program:
	contains(approximate_formats(), 'EPUB|MOBI|PDF', 'red', '')

Why better? Because:

• As I said abive, GPM templates are compiled so the custom column doesn't have that advantage.
• There is no list manipulation in this template.
• It is general enough to return different values for yes and no.

Note that this template suffers from the same problem as the function example I gave. It will return true if a format string contains one of the values. For example, it will match ORIGINAL_EPUB as well as EPUB. If you don't want this then you could use the following:

program:
	list_re(approximate_formats(), ',', '^(EPUB|MOBI|PDF)$', 'red');

but this is significantly slower.

Alternatively you could do the same thing as a custom template function, which will be faster because it isn't as general as list_re.

def evaluate(self, formatter, kwargs, mi, locals, color):
	import re
	fmt_data = mi._proxy_metadata.db_approx_formats
	for f in fmt_data:
		if re.search('^(EPUB|MOBI|PDF)$', f):
			return color
	return ''

[chaley]

Originally Posted by readin

But I'm stuck on this one:

name: icon_if_val
arg count: 3
doc: icon_if_val(field, val, icon) -- if 'field' contains 'val', returns icon filename ('icon' + '.png'), else returns 'blank.png'
program code:
def evaluate(self, formatter, kwargs, mi, locals, field, val, icon):
	import re
	if re.search(val, mi.get(field)):
		return icon + '.png'
	return 'blank.png'

Result:
EXCEPTION: expected string or buffer

You didn't supply a call example, but I can see one potential problem. If 'field' is like tags then the result from mi.get will be an array (in python a 'list'), not a string. If you know that 'field' is always 'like tags' then you can solve this problem using

	import re
	f = ','.join(mi.get(field))
	if re.search(val, f):

If you don't know then you can do something like

	import re
	f = mi.get(field)
	if isinstance(f, list):
		f = ','.join(f)
	if re.search(val, f):

We are getting very technical here.

[readin]

Originally Posted by chaley

We are getting very technical here.

True, but you're very good at explaining!

Here's what I have now:

1) An advanced rule for coloring books if they don't have an ebook format:

program:
	contains(approximate_formats(), 'EPUB|MOBI|PDF', '', 'red')

2) An advanced rule for composing icons which calls three functions:

program:
	strcat(
		icon_if_val('#own', 'p', 'book'),
		':',
		icon_if_set('series', 'series'),
		':',
		icon_if_fmt('LNK', 'folder')
	)

This function checks if a field has a specified value:

name: icon_if_val
arg count: 3
doc: icon_if_val(field, val, icon) -- if 'field' contains 'val', returns icon filename ('icon' + '.png'), else returns 'blank.png'
program code:
def evaluate(self, formatter, kwargs, mi, locals, field, val, icon):
	import re
	f = mi.get(field)
	# if 'field' is an array (like tags), convert it to a CSV list:
	if isinstance(f, list):
		f = ','.join(f)
	if re.search(val, f):
		return icon + '.png'
	return 'blank.png'

This function checks if a field has any value (is not empty):

name: icon_if_set
arg count: 2
doc: icon_if_set(field, icon) -- if 'field' value is set, returns icon filename ('icon' + '.png:'), else returns 'blank.png'
program code:
def evaluate(self, formatter, kwargs, mi, locals, field, icon):
	if mi.get(field):
		return icon + '.png'
	return 'blank.png'

This function checks if a book has a specified format:

name: icon_if_fmt
arg count: 2
doc: icon_if_fmt(format, icon) -- if format is found, returns icon filename ('icon' + '.png'), else returns 'blank.png'
program code:
def evaluate(self, formatter, kwargs, mi, locals, format, icon):
	import re
	fmt_data = mi._proxy_metadata.db_approx_formats
	fmt_data = ','.join(v.upper() for v in fmt_data)
	if re.search(format, fmt_data):
		return icon + '.png'
	return 'blank.png'

As you correctly guessed, the field I'm sending to the icon_if_val() function is like tags. But I used your second example anyway, which checks if the field is an array, in case I want to expand my use of that function in the future.

Everything seems to be working perfectly. Thanks so much for all your help!