Search for books with author notes and show one book per author

[Wiggo]

As I said, the problem was sitting in front of the monitor - I feel dumb again

This opend my eyes

Originally Posted by chaley

This is my saved search, using the stored template:
Attachment 206934

The names must be spelled exactly the same in the stored templates and the saved searches.

I tried it first in advanced search with templates and there it didn't work

[Comfy.n]

Originally Posted by chaley

@Comfy.n: in your chain "One + Has Bio" you should put the "item_has_notes" search term before the "get_one_book_per_field_value" search term. The way you have it the search can miss results for the reasons explained in post 21.

Yes I hadn't noticed there is a difference. I've replaced the attachment, in case someone wants to use the same menu items, renaming that one to Has Bio + One instead of One + Has Bio.

[Comfy.n]

Originally Posted by chaley

This is my saved search, using the stored template:
Attachment 206934

The names must be spelled exactly the same in the stored templates and the saved searches.

I like to see the original search expression on the search bar. That's one of the reasons why I often create menu items in Action Chains and use those instead of saved searches!

[chaley]

Originally Posted by Comfy.n

I like to see the original search expression on the search bar. That's one of the reasons why I often create menu items in Action Chains and use those instead of saved searches!

Interesting. I prefer the brevity. When I want to see the "real" search I use the menu item in the search menu to get the underlying search text. It's good we both can have what we want.

[Comfy.n]

Originally Posted by chaley

This has become a game of whack-a-Mole. The number of variants one can make on this theme is unbounded. Why not show 2 books per series where the author has the letter 'Q' in the name and it doesn't have the tag "mumble"? Or two publishers filtered in reverse alphabetical order when the books have ratings > 4? Or the last 3 books by date from an author with notes?

I won't go that far, I hope!!

[ownedbycats]

Would the method be much different for showing one book per series (with notes), with the lowest-indexed book per series?

[chaley]

Originally Posted by ownedbycats

Would the method be much different for showing one book per series (with notes), with the lowest-indexed book per series?

Are you asking about the stuff I talked about in post 24? If so then the process would be:

1. Run the search
   

   template:"""program: item_has_notes('series')#@#:b:yes"""

   This gets you a list of books where the series value has a note attached.
2. Make a temp VL from this search. This isn't strictly necessary because you could use a compound search, but it does make it easier to sort and to experiment to get the list you want.
3. Sort the booklist by series ascending.
4. Run

   template:"""program: get_N_books_per_field_value('series',1)#@#:b:yes"""

   The template used in this search is still under thought/development.

You would see a list one book per series, where that book has the lowest series index.

If you sorted the series descending in step 3 then you would get the highest numbered book in each series.

If you search for rating:true, sort the rating column in reverse, then run the template in step 4 with 'publisher', you would get the highest rated book for each publisher.

And so on.

[chaley]

After playing for a while I have something that works for me.

The following template processes the books displayed on the book list, in the order they are displayed. The template has two arguments: "field name" and "item count". It looks at "field name" in each book (in order). If the book has an "item value" in that field, it checks if it already has found "item count" books for that "item value". If it has then it ignores the book. If it hasn't then it adds the book to the list of books to display.

Spoiler:

python:
def evaluate(book, context):
	from collections import defaultdict
	if context.arguments is None or len(context.arguments) != 2:
		raise ValueError("This template requires two arguments, field_name and item_count")
	field_name = context.arguments[0]
	desired_count = int(context.arguments[1])

	data_view = context.db.data
	db = context.db.new_api
	# Check if we have already computed the necessary data
	book_ids = context.globals.get('book_ids', None)
	if book_ids is None:
		# The candidates are the books matched by any previous search expression
		candidates = context.globals.get('_candidates', db.all_book_ids())
		books = defaultdict(set)
		answer = set()
		# Scan the library from top to bottom, in the order being displayed. This
		# reflects what is currently shown in the GUI, such as previous searches or VLs.
		for book_id in (tr.book_id for tr in data_view):
			# Check if this book is allowed by previous search terms
			if book_id not in candidates:
				continue
			# Get the value(s) to be checked.
			item_values = db.field_for(field_name, book_id)
			if item_values is None:
				continue
			if isinstance(item_values, str):
				# We have a single value, such as a publisher or series.
				# Make it into a list of one value
				item_values = (item_values,)
			for item_val in item_values:
				# Check if we have seen this item value before.
				item_set = books[item_val]
				if len(item_set) < desired_count:
					# We haven't hit the limit of books with this item value to display
					item_set.add(book_id)
					answer.add(book_id)
		# Save the computed results to be used for the rest of the books being examined
		context.globals['book_ids'] = answer

	# Check if the current book is to be displayed
	book_ids = context.globals['book_ids']
	return '1' if book.id in book_ids else ''

Because the book list is scanned in displayed order, how the book list is sorted is important. The "item values" are checked in their order of appearance. This lets you control which books you get, for example:

• Highest or lowest series index.
• Find series with more than 5 books then show the book with the lowest series index. (Uses this template search for the Find series part.)
• Tags with the highest or lowest rating.
• Publishers with highest or lowest rating.
• Authors of books with a series that has notes, in genre order.
• And so on.

Basically, if you can sort the book list into the order you want, this template will respect that order, showing "item count" books having "field name" with some "item value".

Although not strictly required, the template works much better if running from latest source. If you don't then search caching can get involved, showing you some previous result instead of a current calculation. The change will be in the next release/preview.

The following example will show the first book of each series that has a rating.

• As above, create a stored template containing the template code in the spoiler. Name it whatever you want. I used "summarize_booklist". Ignore the ValueError shown in the Stored templates dialog.
• Use search to find the list of books you want to process. I recommend you make a temporary VL for the search once you have what you want. Example: rating:true
• Sort the book list in the order you want books to be examined. Example: series ascending.
• Using Advanced search, make a template search. Example: lowest series index. Because I used the name "summarize_booklist", the search looks like this:
  
  
  The template is
  

  program: summarize_booklist('series',1)

[Comfy.n]

Originally Posted by chaley

The template is

program: summarize_booklist('series',1)

Looks awesome, I must have missed this post! Will try it when I'm not almost asleep.

edit: Wheee! Just did a quick test with template:"""program: summarize_booklist('publisher',15)#@#:b:yes""" and v:true (v= GR votes), then sorted by multiple columns on Publisher then GR votes, thus getting my listing of top voted titles by Publisher:

[Wiggo]

@chaley

I have an author who has both author notes and series notes.

He is listed in author notes and "one book per author", but unfortunately not in series notes and "one book per author" - series notes only he is listed too.

What additional information would you need, I always forget something?

edit: the author has also aliases

[chaley]

Originally Posted by Wiggo

@chaley

I have an author who has both author notes and series notes.

He is listed in author notes and "one book per author", but unfortunately not in series notes and "one book per author" - series notes only he is listed too.

What additional information would you need, I always forget something?

edit: the author has also aliases

I don't understand the question. What exactly are you searching for (the search string)? And how is the booklist sorted? Remember, the template keeps the first item it encounters. In your case I suspect the first author isn't the one with notes.

The template knows nothing of aliases. It compares the author names, and either they are the same or they aren't.

[Wiggo]

Originally Posted by chaley

I don't understand the question. What exactly are you searching for (the search string)? And how is the booklist sorted? Remember, the template keeps the first item it encounters. In your case I suspect the first author isn't the one with notes.

The template knows nothing of aliases. It compares the author names, and either they are the same or they aren't.

First I click in the tag browser on my saved search "has series notes"



and then additionally on "one book per author"



and Dan Adams disappears. What am I doing wrong?

[Comfy.n]

Hi Wiggo, I liked your icons for book formats. They fit better on dark theme than the ones I'm using. Would you mind sharing them in a zip attachment?

I'd never thought of using a column just for symbols, seems a good thing to do!

[chaley]

Originally Posted by Wiggo

First I click in the tag browser on my saved search "has series notes"

Attachment 207378 Attachment 207379

and then additionally on "one book per author"

Attachment 207380 Attachment 207381

and Dan Adams disappears. What am I doing wrong?

My guess is that "Dan Adams" is the author of books in a different series or no series, and one of those books is appearing ahead of the books with series with a note.

Order is important in "and" search clauses. Clauses following an "and" operate of the results of the previous search clauses. As such they don't see any books already filtered out. This "shortcutting" improves search performance a lot.

Try changing the order of the search to

search:"=#Has.Series notes" and search:"=#Has._One book per author"

This order ensures that the one-book-per-author search operates on the results of the has-series-notes search.

[ownedbycats]

Originally Posted by Comfy.n

I'd never thought of using a column just for symbols, seems a good thing to do!

And here I thought I was being creative by doing this to the Formats column: