Search for books with author notes and show one book per author

[Comfy.n]

Originally Posted by chaley

You could also do things like

not tag:foobar and search:one_book_per_author

producing a list of one book per author where the books don't have the foobar tag.

Do I have this right?

Exactly. I haven't used the 'and' connector in my posts because that's how I always do the searches, given they behave the same when 'and' is omitted

[Comfy.n]

Originally Posted by chaley

Try this to see if I understand you correctly.

Should I use the same settings as with the previous one, I mean

Template value field: yes
Comparison type: Set/Not set

I've tried that way and it's running indefinitely on a small VL

[Comfy.n]

By the way, I find Calibre search amazing. It can even understand parentheses syntax, as in:

l:en (f:ep or f:az or f:mob) tag:non

where l is my short search alias for languages, f is for formats.

ep is my 'epub' lazy form

[chaley]

I may have confused things, but lets put that aside.

I have an implementation that doesn't require the temporary VL but respects the current VL. It is a bit more complicated because I wanted to generalize it to any field, not just authors. To use what I describe below you must be running calibre 7.7 or later.

To use it, make a stored template named whatever you want -- I use "get_one_book_per_field_value". The template itself is:

Spoiler:

python:
def evaluate(book, context):
	field_name = context.arguments[0] if len(context.arguments) == 1 else None
	if field_name is None:
		raise NameError("The argument 'field' wasn't provided")

	db = context.db.new_api
	# Check if we have already computed the necessary data
	book_ids = context.globals.get('book_ids', None)
	if book_ids is None:
		candidates = context.globals.get('_candidates', db.all_book_ids())
		books = {}
		for book_id in db.search(query=''): # restrict the search to the current VL
			if book_id not in candidates:
				continue
			item_values = db.field_for(field_name, book_id)
			if item_values is None:
				continue
			if isinstance(item_values, str):
				item_values = (item_values,)
			for item_val in item_values:
				if item_val not in books or books[item_val]['count'] > len(item_values):
					books.pop(item_val, None)
					books[item_val] = {'count': len(item_values), 'book_id': book_id}
		context.globals['book_ids'] = frozenset(bn['book_id'] for bn in books.values())

	# Check if the current book is to be displayed
	book_ids = context.globals['book_ids']
	return '1' if book.id in book_ids else ''

This template takes one argument, the field (column) you are interested in (authors).

Next, make this template search:

template:"""program: get_one_book_per_field_value('authors')#@#:b:yes"""

It calls the above stored template with the argument "authors". I recommend you create a saved search with it. I did and named it "one_book_per_author". You now have @Katja_hbg's B.

You also need @Katja_hbg's A. To get it, put this in a stored template named what you want. I named it "item_has_notes". The template is

Spoiler:

python:
def evaluate(book, context):
	field_name = context.arguments[0] if len(context.arguments) == 1 else None
	if field_name is None:
		raise NameError("The argument for the field name wasn't provided")

	db = context.db.new_api
	items_with_notes = context.globals.get('items_with_notes')
	if items_with_notes is None:
		items_with_notes = db.get_all_items_that_have_notes(field_name)
		context.globals['items_with_notes'] = items_with_notes
		context.globals['item_name_map'] = db.get_item_name_map(field_name)
	item_name_map = context.globals['item_name_map']

	vals = book.get(field_name)
	if vals is None:
		return ''
	if isinstance(vals, str):
		vals = tuple((vals,))
	for val in vals:
		if item_name_map.get(val) in items_with_notes:
			return '1'
	return ''

This template also takes 1 argument, the field (column) name.

Now make another template search

template:"""program: item_has_notes('authors')#@#:b:yes"""

This calles the stored template with the field name "authors". Again, I recommend you save it as a search. I did that, naming it "author has notes".

You can now do searches like:

• search:author has notes and search:one_book_per_author
• formats:epub and tags:foobar and search:one_book_per_author
• search:one_book_per_author

All of this works for me. I hope it works for you.

[Comfy.n]

Originally Posted by chaley

All of this works for me. I hope it works for you.

Yes, that works perfectly! Also, surprisingly fast.

Running from source, I've just tested

template:"""program: get_one_book_per_field_value('authors')#@#:b:yes"" " tag:fantasy

template:"""program: get_one_book_per_field_value('authors')#@#:b:yes"" " tag:biography f:ep

Both return only one title per author.



edit to add: it works also on latest preview version, by the way!

[chaley]

Originally Posted by Comfy.n

Yes, that works perfectly! Also, surprisingly fast.

Running from source, I've just tested

template:"""program: get_one_book_per_field_value('authors')#@#:b:yes"" " tag:fantasy

template:"""program: get_one_book_per_field_value('authors')#@#:b:yes"" " tag:biography f:ep

Both return only one title per author.

Running from source, it would be significantly faster to have the get_one_book ... as the last expression in the search. Reason: the change I made to base calibre is to pass to the template the set of books matching the previous expressions, allowing the get_one_book... analysis to operate on only those books.

In addition, in some cases it is possible for the get_one_book ... to select a book for an author that won't match the rest of the search expression. Were this to happen, the answer would be missing that author. For example, imagine that Joe Blogs has two titles, A and B. Title A is tagged "Historical Fiction". Title B is tagged "Science Fiction". The get_one_book ... algorithm could pick either book as the "one title". If you combine it with a tags:... clause then the tag might match the other book, resulting in neither book being found. This won't happen if get_one_book ... is the last expression in the search and you are using the latest source.

edit to add: it works also on latest preview version, by the way!

Yes, with a) significantly worse performance because it operates in the entire library and b) increased probability of selecting a "one title" that isn't matched by the rest of the search.

[chaley]

The calibre change mentioned above is in release 7.7, released today.

[Comfy.n]

Inspired by this question, today I decided to use publisher instead of authors:

template:"""program: get_one_book_per_field_value('publisher')#@#:b:yes"""

This suggested me an interesting possibility, as I have the Goodreads Metadata plugin: to sort a library by most voted titles, getting only one book by publisher...

In my mind, what followed was: what if I could display only like 5 or 10 titles per publisher? Then I would be able to see filtered in a library a list of the top voted books by publisher!

[chaley]

Originally Posted by Comfy.n

Inspired by this question, today I decided to use publisher instead of authors:

template:"""program: get_one_book_per_field_value('publisher')#@#:b:yes"""

This suggested me an interesting possibility, as I have the Goodreads Metadata plugin: to sort a library by most voted titles, getting only one book by publisher...

In my mind, what followed was: what if I could display only like 5 or 10 titles per publisher? Then I would be able to see filtered in a library a list of the top voted books by publisher!

This has become a game of whack-a-Mole. The number of variants one can make on this theme is unbounded. Why not show 2 books per series where the author has the letter 'Q' in the name and it doesn't have the tag "mumble"? Or two publishers filtered in reverse alphabetical order when the books have ratings > 4? Or the last 3 books by date from an author with notes?

Other templates I have already written are further variants. Example, show the last book in a series and show the first book in a series.

At first blush I don't want to go here. However, I do have an idea that might work and be of general use -- iterate through the displayed books (the current book list) in order, showing N books per field. To use it, one would search for the books of interest and make a temp VL of that search. Then sort the booklist in the order you want the filtering. Then run the filter. This could do all of the above, but does take some thought and planning to make work. I need to think about this.

[Wiggo]

Unfortunately, the one in post #19 does not work for me.
Both templates already show the same error (Calibre 7.7)



But my feeling tells me that the problem is in front of the computer.

[chaley]

Originally Posted by Wiggo

Unfortunately, the one in post #19 does not work for me.
Both templates already show the same error (Calibre 7.7)

But my feeling tells me that the problem is in front of the computer.

Those stored templates won't work in the template tester because they require an argument, the column name. That is why I supplied the template searches, which provide the needed argument to the stored templates.

You can also test the stored templates using the "Test" button in the Stored template dialog, but it isn't much of a test as it looks at only the currently selected books. For example, it you press the Test button and enter

get_one_book_per_field_value('authors')

you will see a 1 or an empty string for each book.

[Wiggo]

I have already tried that template searches




The same happens with

template:"""program: get_one_book_per_field_value('authors')#@#:b:yes"""

[Comfy.n]

Originally Posted by Wiggo

Unfortunately, the one in post #19 does not work for me.
Both templates already show the same error (Calibre 7.7)

Hi! The stored templates looked the same here, in the template tester. I just ignored that message. I've attached my Action Chains menu items for these searches, but they will only work if the stored templates are already present with the same names @chaley suggested: item_has_notes or get_one_book_per_field_value

[chaley]

Originally Posted by Wiggo

I have already tried that template searches

Attachment 206931
Attachment 206932

I assume you are saying they didn't work. What did they do?

Here are my stored templates. They show up in the stored templates dialog, like this:


This is my saved search, using the stored template:


The names must be spelled exactly the same in the stored templates and the saved searches.

[chaley]

Originally Posted by Comfy.n

Hi! The stored templates looked the same here, in the template tester. I just ignored that message. I've attached my Action Chains menu items for these searches, but they will only work if the stored templates are already present with the same names @chaley suggested: item_has_notes or get_one_book_per_field_value

@Comfy.n: in your chain "One + Has Bio" you should put the "item_has_notes" search term before the "get_one_book_per_field_value" search term. The way you have it the search can miss results for the reasons explained in post 21.