Search for books with author notes and show one book per author

[chaley]

EDIT: See this post for an updated & generalized version of the following template search.

@Comfy.n asked me if there was a template search that for each author with a note would show one book by that author. It didn't matter which book. Here is that search.

Showing one book per author might not be possible if the author is a coauthor on several books. For example, consider these books where All of A, B, and C have notes:

• Title 1, authors A & B
• Title 2, authors C & B

Title 1 and Title 2 will both appear in the search results because there is no other way to show the authors A and C. This means that two books by author B will appear.

The search template gives priority to books with single authors. Extending the above example:

• Title 1, authors A & B
• Title 2, authors C & B
• Title 3, author B
• Title 4, author C

results in

• Title 1 (only book with author A)
• Title 3 (author B is sole author)
• Title 4 (author C is sole author)

Title 3 is shown even though B is a coauthor of Title 1.

The template can easily be changed to check any field, for example series, tags, or some custom column.

The search respects virtual libraries.

Use the Advanced search / Template search to create the search.
Template value field: yes
Comparison type: Set/Not set
Template:

Spoiler:

python:
def evaluate(book, context):
	# Set this field to the lookup name of the desired field.
	field_name = 'authors'

	db = context.db.new_api
	# Check if we have already computed the necessary data
	book_ids = context.globals.get('book_ids', None)
	if book_ids is None:
		book_notes = {}
		all_notes = db.get_all_items_that_have_notes(field_name)
		for book_id in db.search(query=''): # restrict the search to the current VL
			item_values = db.field_for(field_name, book_id)
			if item_values is None:
				continue
			if isinstance(item_values, str):
				item_values = (item_values,)
			for item_val in item_values:
				item_id = db.get_item_id(field_name, item_val)
				if item_id in all_notes:
					if item_val not in book_notes or book_notes[item_val]['count'] > len(item_values):
						book_notes[item_val] = {'count': len(item_values), 'book_id': book_id}
		context.globals['book_ids'] = frozenset(bn['book_id'] for bn in book_notes.values())

	# Check if the current book is to be displayed
	book_ids = context.globals['book_ids']
	return '1' if book.id in book_ids else ''

Here is an image of the filled-in Advanced search dialog.

[Comfy.n]

Thanks so much, chaley! One of the reasons for this odd search is that it makes it easier to preview distinct author notes in the Book Details pane, which I have on the right side of the UI.

Now I'm thinking I could use a trimmed Comments custom column to be displayed above the Bio. So I would need one more tip: what template should I use to display a trimmed comments field above the author bio in BD, displaying only the first 300 characters from the Comments built in column?

I gather from this thread that I could use something like:

{comments:shorten(300,...,0)}

Is that correct and also, would a PTM template be more suitable performace-wise?

[chaley]

Originally Posted by Comfy.n

Thanks so much, chaley! One of the reasons for this odd search is that it makes it easier to preview distinct author notes in the Book Details pane, which I have on the right side of the UI.

Now I'm thinking I could use a trimmed Comments custom column to be displayed above the Bio. So I would need one more tip: what template should I use to display a trimmed comments field above the author bio in BD, displaying only the first 300 characters from the Comments built in column?

I gather from this thread that I could use something like:

{comments:shorten(300,...,0)}
Is that correct?

That works fine. You could also do it in GPM using

program: shorten($comments, 300, '...', 0)

I don't think that the performance of the two will be significantly different, especially if the shortened column is displayed only in book details.

Would a PTM template be more suitable performace-wise?

Performance isn't an issue if the column is displayed only in book details. And in any event, for something this simple I don't think the difference between any of the modes will be significant.

[Comfy.n]

That works smoothly.

I'm using the GPM one, and after creating the column with the option "Show as HTML in BD" unchecked, it would display the text with the blue clickable option to "Search all books with..." Then I left "show as html.." checked and it displayed as plain text, like I wanted.

[Comfy.n]

Originally Posted by chaley

@Comfy.n asked me if there was a template search that for each author with a note would show one book by that author. It didn't matter which book. Here is that search.

It takes a while to refresh the view if changing metadata while the query is active (whilst it's visible on the search bar, I mean), so I'm tagging temporarily the results as 'temp', then only changing metadata on the results after doing a new search for tag:temp.

[chaley]

Originally Posted by Comfy.n

It takes a while to refresh the view...

That doesn't surprise me. The search looks at every book on the library to build an index of authors with notes, which requires multiple probes of the notes database. It is a very expensive compute- and disk-bound process.

Using a temporary tag is a good workaround.

[Comfy.n]

This is off-topic, but I think you may know something about this issue.

In 2022, I was able to somewhat style the scrollbar for the Dark Theme. It was not perfect at all, but provided good visibility for the handle. Nowadays I'm used to do just keyboard commands like 'home' in cover grid view, and just forget the scrollbar. Also, that only worked with the system default UI style.

With the recent addition of a more refined color customisation dialog, I've tried today two things:

1) try more combinations of colors in the dark theme for the currently customisable UI elements, but that didn't work well

2) I've tried runnning from source again and adding that code Kovid suggested in 2022, but the change wouldn't take. (that was args = sys.argv[:1] args += ['-stylesheet', 'test1.qss'])

Some Qt based programs have a different scrollbar style in dark theme, where the handle has more like a light gray tone and the rest is black, so I wonder if that could somehow be tweaked in Calibre, with the recent Qt update.

[chaley]

Sorry, but I have no idea.

[Comfy.n]

Hi again,

Today I missed a way to restrict a search with the second part of this request. For example, a search for

tags:biography listing_restriction:one book per author.

Another one would be:

 languages:eng formats:epub listing_restriction:one book per author.

Can that second part be written as a separate query, so one could run it, say, on a VL?

[chaley]

Originally Posted by Comfy.n

Hi again,

Today I missed a way to restrict a search with the second part of this request. For example, a search for

tags:biography listing_restriction:one book per author.

Another one would be:

 languages:eng formats:epub listing_restriction:one book per author.

Can that second part be written as a separate query, so one could run it, say, on a VL?

I don't understand. What does

listing_restriction:one book per author

mean?

The template search already respects the current VL. If you want it to respect some VL when no VL is currently applied, put that VL's search into the expression. For example, assuming the VL search expression is a saved search named "vl_search":

search:vl_search and tags:foo and search:one_book_per_author

Hmmmm ... it might be that the template search skips books because it selects a book outside the current VL to be displayed, then rejects it later when it isn't in the VL. Is this the problem you are describing?

[Comfy.n]

The original request included filtering titles with author notes.

I can't use that to restrict a search for titles in my library that contain the tag:biography but don't contain author notes.

I'm running the Extract People Metadata plugin on my libraries, which requires a lot of tweaking to get translators, original titles etc. and restricting any search to only one book per author would make it easier to spot the troublesome books for extracting that info...

edit to add:

As I see my original question, this is what it meant:

a + b

being

a = has author notes
b = restrict results to only one title per author

And it does exactly that, however it does it in a way that B depends on A (or would it be A depending on B to list the results? I can't tell)

Now, in my mind, if B could be run "independently", then I could do b + c, b + d, b + e etc. c, d and e being any other search. Maybe not possible, I suspect.

[Katja_hbg]

Forme it sounds that you are looking for

b = restrict results to only one title per author (without any further filter)
and in addition - optional i.e. can be switched on/off:
a = has author notes (which is in fact a subset of "b")

[chaley]

I think you are asking that the template pay no attention to notes. Instead it takes a set of books and reduces that set to one book per author.

If true, then this pseudo search would do the same as the original template:

search:has_author_notes and search:one_book_per_author

You could also do things like

not tag:foobar and search:one_book_per_author

producing a list of one book per author where the books don't have the foobar tag.

Do I have this right?

[chaley]

Try this to see if I understand you correctly.

1. Create a VL using the search you want. A temporary VL (current search) works.
2. Run this template in the template search
   
   Spoiler:

   python:
   def evaluate(book, context):
   	# Set this field to the lookup name of the desired field.
   	field_name = 'authors'
   
   	db = context.db.new_api
   	# Check if we have already computed the necessary data
   	book_ids = context.globals.get('book_ids', None)
   	if book_ids is None:
   		books = {}
   		# all_notes = db.get_all_items_that_have_notes(field_name)
   		for book_id in db.search(query=''): # restrict the search to the current VL
   			item_values = db.field_for(field_name, book_id)
   			if item_values is None:
   				continue
   			if isinstance(item_values, str):
   				item_values = (item_values,)
   			for item_val in item_values:
   				item_id = db.get_item_id(field_name, item_val)
   				if item_val not in books or books[item_val]['count'] > len(item_values):
   					books[item_val] = {'count': len(item_values), 'book_id': book_id}
   		context.globals['book_ids'] = frozenset(bn['book_id'] for bn in books.values())
   
   	# Check if the current book is to be displayed
   	book_ids = context.globals['book_ids']
   	return '1' if book.id in book_ids else ''

It should produce a list of one book per author where the books are limited to those in the VL.

If this is what you want then I will look at ways to eliminate the temporary VL.

[Comfy.n]

Originally Posted by Katja_hbg

Forme it sounds that you are looking for

b = restrict results to only one title per author (without any further filter)
and in addition - optional i.e. can be switched on/off:
a = has author notes (which is in fact a subset of "b")

yes, and this reminds me that occasionally I might want to run just B on a whole library...