[chaley]

I did some tests on my 77,500-book library.

With no data files at all, repeated runs took an average of 18.5 seconds. There was one outlier that took 30 seconds, which I think happened because my cloud auto-backup daemon kicked in.

With 40, 50, and 60 data files it took an average of 18.7 seconds, with no significant outliers.

I tried this python template that marks matched books instead of using search. It is slower, taking around 1.5 seconds longer than the search. I am sure that the difference is in the GUI refresh to show the marks, but I didn't do the work to verify it.

python:
def evaluate(book, context):
	from calibre.db.constants import DATA_FILE_PATTERN
    from time import monotonic
    start = monotonic()
	count = 0;
    db = context.db
    db.data.set_marked_ids(set())
    db = db.new_api
    found = set()
	for id_ in db.all_book_ids():
		files = tuple(f.relpath.partition('/')[-1] for f in
			db.list_extra_files(id_, use_cache=True, pattern=DATA_FILE_PATTERN))
		if len(files) > 0:
			#print(f'{id_}: {files}')
			count += 1
            found.add(id_)
    context.db.data.set_marked_ids(found)
	return (f'template search time: {monotonic() - start:.2f}, {count}')

All these tests are on Windows 10 Pro, 16GB Ram, an Intel Core i7-10710U CPU, a 512GB SSD, MS Defender A/V, and current calibre source running in debug mode. I have the following plugins installed:


Bottom line: I can't reproduce a hang. The process can take several seconds. I think the process will slow down significantly if the computer is memory-challenged or if there are other processes competing for the CPU. In particular, an anti-virus that monitors file accesses could greatly affect performance because multiple accesses are required to check each book.