Is there a way to get author/tag/series/publisher ids+text within a virtual library?

[kiwidude]

In my Find Duplicates plugin, the main "Find Book Duplicates" functionality respects whatever virtual library search restriction you have applied.

EDIT: On further reading I can see that it uses self.gui.library_view.model() to operate within, and then only where it needs a more specific subset that it uses db.data.search_getting_ids() passing the restriction through.

However the Metadata Variations duplicate check does *not* currently respect the virtual library selection. Looking at the code I wrote forever ago, it seems that for those searches I am invoking one of the following depending on user selection:

• db.get_authors_with_ids()
• db.get_series_with_ids()
• db.get_publishers_with_ids()
• db.get_tags_with_ids()

Is there an API equivalent I could use for these that could take a search_restriction argument? Or should I just iterate over the model to build up my own dictionaries?

[kovidgoyal]

Not sure what you are asking for exactly, but you first get the set of all book ids in the current virtual library. THen after that call

ans = set()
for book_id in book_ids:
   ans |= {x for x in db.new_api.field_ids_for('tags', book_id)}

and use get_id_map() to convert the ids from ans into names.

[kiwidude]

Originally Posted by kovidgoyal

Not sure what you are asking for exactly, but you first get the set of all book ids in the current virtual library. THen after that call

ans = set()
for book_id in book_ids:
   ans |= {x for x in db.new_api.field_ids_for('tags', book_id)}

and use get_id_map() to convert the ids from ans into names.

Hi Kovid - thanks for the reply. What I wanted was exactly the same output that the direct db functions above give me, of pairs of ids with their text value. e.g. all the unique author id + their name combinations.

So it appears that indeed the answer is I need to iterate over all the books and build up my own (unique) sets...

[chaley]

I think you want something like this ids_for_field() function. I don't know if the order is right. This produces (id, name) but you might want (name, id).

python:

def ids_for_field(db, ids_of_books, field_name):
	# First get all the names for the desired books.
	# Use a set to make them unique
	unique_names = set()
	for tup in db.all_field_for(field_name, ids_of_books).values():
		for vals in tup:
			unique_names.update((vals,))
	# Now get the ids for the names and build the pairs
	id_field_pairs = list()
	for aut in unique_names:
		id_field_pairs.append((db.get_item_id(field_name, aut), aut))
	return id_field_pairs

def evaluate(book, context):
	db = context.db.new_api
	print('--------------------')
	# Get the list of books in the current VL
	ids_in_vl = db.search('', restriction='authors:a')
	# Get the id,val pairs for the desired field
	field_pairs = ids_for_field(db, ids_in_vl, 'authors')
	print(field_pairs) 
	return ''

NB: I am using the python template facility to test the code, which is why "python:" and "def evaluate()" are there. It hadn't occurred to me when we added it that one use is interactive python code debugging inside calibre.

[chaley]

Here is an example that uses the current VL

python:

def ids_for_field(db, ids_of_books, field_name):
	# First get all the names for the desired books.
	# Use a set to make them unique
	unique_names = set()
	for tup in db.all_field_for(field_name, ids_of_books).values():
		for val in tup:
			unique_names.add(val)
	# Now get the ids for the names and build the pairs
	id_field_pairs = list()
	for name in unique_names:
		id_field_pairs.append((db.get_item_id(field_name, name), name))
	return id_field_pairs

def evaluate(book, context):
	print('--------------------')
	# Get the list of books in the current VL
	ids_in_vl = context.db.data.search_getting_ids('', '', use_virtual_library=True)
	# Get the id,val pairs for the desired field
	field_pairs = ids_for_field(context.db.new_api, ids_in_vl, 'authors')
	print(field_pairs) 
	return ''

[kiwidude]

Awesome, thanks chaley! That will save me much time in experimenting

[chaley]

Apologies for the churn. I looked at the underlying code for the API methods I used and found a significant inefficiency in what I proposed. The underlying reversed dictionary {id:name} was being generated for *every* field item. For big libraries this could be a significant performance hit.

Here is a version that reverses the id map once then uses it to build the output.

def ids_for_field(db, ids_of_books, field_name):
	# First get all the names for the desired books.
	# Use a set to make them unique
	unique_names = set()
	for tup in db.all_field_for(field_name, ids_of_books).values():
		for val in tup:
			unique_names.add(val)
	# reverse the map of ids to names so id_map[name] gives the id
	id_map = {v:k for k,v in db.get_id_map(field_name).items()}
	# Now build the pairs (id, name)
	id_field_pairs = list()
	for name in unique_names:
		id_field_pairs.append((id_map[name], name))
	return id_field_pairs

[kiwidude]

Oh that one made a massive difference indeed on my local library - thanks chaley!