Plugboard, template, and custom composite column recipes

[chaley]

Originally Posted by Protoplast26

Just a quick question

Is it possible to change the save template on books that are already on the device, meaning books that ware using old template be changed to a new templete without having to resend the book?

If I understand the question, you want to rename books already on the device by changing the template. This isn't possible with calibre. You must manually delete the old copy then send the new one. Annotations and similar associated information will probably be lost.

[Mister L]

I would like to automatically add books from a specific publisher to a Kobo collection, but only for certain publishers, not all. So I am trying to make a plugboard rule that says "if publisher is X then add book to collection Y" (and if publisher is not X then do nothing). I have made a custom column for the Kobo collections which is called #kobo. I have been looking in the user manual and I think this is possible using advanced formatting, based on the example given for series, but I cannot figure out how to do it...

Any help much appreciated (including confirmation it is not actually possible, so I can stop trying to figure out how. ).

[chaley]

Originally Posted by Mister L

I would like to automatically add books from a specific publisher to a Kobo collection, but only for certain publishers, not all. So I am trying to make a plugboard rule that says "if publisher is X then add book to collection Y" (and if publisher is not X then do nothing). I have made a custom column for the Kobo collections which is called #kobo. I have been looking in the user manual and I think this is possible using advanced formatting, based on the example given for series, but I cannot figure out how to do it...

Any help much appreciated (including confirmation it is not actually possible, so I can stop trying to figure out how. ).

Although it is possible to do this in a plugboard, it is the most difficult approach.

The right approach is to use a custom "column built from other columns" where the template computes the collection name. You don't say whether the collection value is a constant, the name of the publisher, or something else. If it is the name of the publisher then a template like this could work:

program:
	str_in_list('Baen, Daw, Orbit, Tom Doherty Associates', ',', $publisher, $publisher, '')

EDIT: You could do it without a custom column by using the "Collections Template" option in the Kobo driver. Depending on your "rules" the template above could work.

[Mister L]

Originally Posted by chaley

Although it is possible to do this in a plugboard, it is the most difficult approach.

Glad I decided to ask for help.

Originally Posted by chaley

The right approach is to use a custom "column built from other columns" where the template computes the collection name. You don't say whether the collection value is a constant, the name of the publisher, or something else. If it is the name of the publisher then a template like this could work:

program:
	str_in_list('Baen, Daw, Orbit, Tom Doherty Associates', ',', $publisher, $publisher, '')

EDIT: You could do it without a custom column by using the "Collections Template" option in the Kobo driver. Depending on your "rules" the template above could work.

Thank you! I think the Collections template could be perfect, if I can use it on top of the other collections columns I already use?

If it's easiest, I could use the publisher name for the collection and change the books I have marked manually, but I would like to keep the collections I already was using if I can.

One publisher name is "BEL Editions" and the collection name is "BEL", and another one is "Bibebook" and the collection is "dp". Would the template look like this?

program:
	str_in_list('BEL Editions, Bibebook', ',', $BEL, $dp, '')

Or do I need to make one rule per collection?

program:
	str_in_list('BEL Editions', ',', $BEL, '')
        str_in_list('Bibebook', ',', $dp, '')

[chaley]

Originally Posted by Mister L

Glad I decided to ask for help.


Thank you! I think the Collections template could be perfect, if I can use it on top of the other collections columns I already use?

Yes, you can use template-based collections with column-based collections.

If it's easiest, I could use the publisher name for the collection and change the books I have marked manually, but I would like to keep the collections I already was using if I can.

One publisher name is "BEL Editions" and the collection name is "BEL", and another one is "Bibebook" and the collection is "dp". Would the template look like this?

program:
	str_in_list('BEL Editions, Bibebook', ',', $BEL, $dp, '')

Or do I need to make one rule per collection?

program:
	str_in_list('BEL Editions', ',', $BEL, '')
        str_in_list('Bibebook', ',', $dp, '')

You need a "rule" per publisher. For me, the clearest and easiest way is this:

program:
	switch_if(
		$publisher == 'BEL Editions', 'BEL',
		$publisher == 'Bibebook', 'dp',
		$publisher == 'Baen', 'Baen',
		$publisher == 'Tom Doherty Associates', 'TOR',
		'')

Notes:

• You can have as many "rules" as you want.
• The equals comparison is case insensitive.
• The '' at the end is the default if no comparison matches. You might want to use an actual value there.
• Having a collection name appear twice, for example in both a column and in the template result, isn't a problem.

[Mister L]

Originally Posted by chaley

Yes, you can use template-based collections with column-based collections.
You need a "rule" per publisher. For me, the clearest and easiest way is this:

program:
	switch_if(
		$publisher == 'BEL Editions', 'BEL',
		$publisher == 'Bibebook', 'dp',
		$publisher == 'Baen', 'Baen',
		$publisher == 'Tom Doherty Associates', 'TOR',
		'')

Notes:

• You can have as many "rules" as you want.
• The equals comparison is case insensitive.
• The '' at the end is the default if no comparison matches. You might want to use an actual value there.
• Having a collection name appear twice, for example in both a column and in the template result, isn't a problem.

Brilliant! That's exactly what I need. Thank you very much for your help.

[BrandonGiesing]

I currently have the Kobo driver set up so anything in the default "tags" column becomes a collection which works fine for the most part but my Kobo takes like 2 minutes to load the Collections page cause I have 5000+ tags (I know, I need to go through and fix things)

I was wondering if there was anyway to use a template to read the item count of the tags so it would only create a collection if a tag has been used like at least ~5+ times and ignore tags below that, that should eliminate a good chunk of the 5000+ and keep the most important ones.

[chaley]

Originally Posted by BrandonGiesing

I currently have the Kobo driver set up so anything in the default "tags" column becomes a collection which works fine for the most part but my Kobo takes like 2 minutes to load the Collections page cause I have 5000+ tags (I know, I need to go through and fix things)

I was wondering if there was anyway to use a template to read the item count of the tags so it would only create a collection if a tag has been used like at least ~5+ times and ignore tags below that, that should eliminate a good chunk of the 5000+ and keep the most important ones.

Yes, this can be done. But to pontificate a bit before going there, I think this is a waste of time. What is "important"? Given spelling variations such as "Science Fiction", "SF, "SciFi", "Sci-Fi", "Science Fiction and Fantasy", and the like; there is no reason to believe that the limited set of tags is useful information. Example, Gibson's work can be called "Science Fiction - Cyberpunk", "Cyberpunk", "SF Cyberpunk", etc. These tags could easily disappear. What I would do is to first use "Manage tags" and fix spelling variations, which can be done fairly quickly. Better would be to build a hierarchical system that makes sense to you then build collections from the first N levels, where N makes sense to you. This second is what I actually do.

Pontificating aside, one can do what you want with a template-language template but with 5,000 tags it would be excruciatingly slow. This python template doesn't have that problem.

python:
def evaluate(book, context):
	field = 'tags'

	# Get the previously computed set of acceptable items, if it exists
	all_names_over_count = context.globals.get('all_names_over_count')
	if all_names_over_count is None:
		db = context.db.new_api
		all_names_over_count = set()
		counts_by_item = db.get_usage_count_by_id(field)
		for item,count in counts_by_item.items():
			if count > 5:
				all_names_over_count.add(db.get_item_name(field, item))
		context.globals['all_names_over_count'] = all_names_over_count

	# Check if the current book has any of the acceptable items
	item_names_in_book = []
	for name in book.get(field):
		if name in all_names_over_count:
			item_names_in_book.append(name)
	return ':@:'.join(item_names_in_book)

[theducks]

Everything Chaley said (BTW the examples were the first group I had to clean up.) My Hierarchy choice was the Major class of the work and the dot part, the REFINED partition. Science Fiction.Military (not the reverse)
Searching on either term will find members. No need for alternate tags (like in that old card file)

You need a PLAN, then fix what you have. Standardize tags. Many of the variation were holdover mentality fro days past where there was a card catalog in the room.

Calibre has a Tag Mapper, that can be used when ADDING BOOKS (Preferences: Adding books: Adding Actions (a tab): control... )

AFAIK this does NOT affect downloaded metadata <hint hint: option to use > or existing books

Another Fix I do is reset Plurals to a singular (Detectives -> Detective ).
I also separate double groups: Science Fiction & Fantasy becomes Science Fiction, Fantasy (or I drop the one Manually when it does not really apply )
Even with all this, I have way too many tags and researching for this, I need to do another house cleaning

[BrandonGiesing]

I know having "proper" tags would be better but this is basically just a stop gap to actually have a usable Kobo until I can get time to go through all like 3000+ books I have A lot of the books are from the FanFicFare plugin so I'm manually adding a lot of items to the "add_to_replace_metadata" section of the personal.ini to cleanup stuff when I notice it to prevent future issues.

Sites I download with that plugin have a lot of crazy tags that are super niche and might only be on one single book in my library but they also often have tags that are more common appearing on many books and often said tags might imply the same thing the rare niche tag did (just more generalized). And then of course, there's the authors that use tags basically for an author note with tags that are like a whole sentence long and I definitely don't need at all.

Hence why I figured "only making collections for tags with over X stories" would help out a ton since it would basically eliminate like 80% of the collections yet books would most likely still appear in a similar collection meaning the same thing.

[BrandonGiesing]

Originally Posted by chaley

Pontificating aside, one can do what you want with a template-language template but with 5,000 tags it would be excruciatingly slow. This python template doesn't have that problem.

python:
def evaluate(book, context):
	field = 'tags'

	# Get the previously computed set of acceptable items, if it exists
	all_names_over_count = context.globals.get('all_names_over_count')
	if all_names_over_count is None:
		db = context.db.new_api
		all_names_over_count = set()
		counts_by_item = db.get_usage_count_by_id(field)
		for item,count in counts_by_item.items():
			if count > 5:
				all_names_over_count.add(db.get_item_name(field, item))
		context.globals['all_names_over_count'] = all_names_over_count

	# Check if the current book has any of the acceptable items
	item_names_in_book = []
	for name in book.get(field):
		if name in all_names_over_count:
			item_names_in_book.append(name)
	return ':@:'.join(item_names_in_book)

Code isn't working, throws the error:

EXCEPTION: Error in function evaluate on line 8 : AttributeError - 'NoneType' object has no attribute 'new_api'

[chaley]

Originally Posted by BrandonGiesing

Code isn't working, throws the error:

Use this:

python:
def evaluate(book, context):
	field = 'tags'

	# Get the previously computed set of acceptable items, if it exists
	all_names_over_count = context.globals.get('all_names_over_count')
	if all_names_over_count is None:
		from calibre.gui2.ui import get_gui
		db = get_gui().current_db.new_api
		all_names_over_count = set()
		counts_by_item = db.get_usage_count_by_id(field)
		for item,count in counts_by_item.items():
			if count > 5:
				all_names_over_count.add(db.get_item_name(field, item))
		context.globals['all_names_over_count'] = all_names_over_count

	# Check if the current book has any of the acceptable items
	item_names_in_book = []
	for name in book.get(field):
		if name in all_names_over_count:
			item_names_in_book.append(name)
	return ':@:'.join(item_names_in_book)

[BrandonGiesing]

Originally Posted by chaley

Use this:

Yep that worked, had to manually edit the Database first though with a SQLite editor to remove all the old entries (Couldn't find any other way to delete all existing collections since Calibre seems to only remove empty ones when you delete books off your device)

Once I did that though, it worked just fine, only tags with 5+ books now have a collection and my device no longer hangs for like 2 minutes loading the collections page. Still takes a solid ~20 seconds but WAY better than before. Should be useable enough while I work to trim down my tags in the meantime.

[theducks]

You can Manage the collections column: the (multi)select the collection name: click the delete icon

[BrandonGiesing]

Originally Posted by theducks

You can Manage the collections column: the (multi)select the collection name: click the delete icon

Tried that like 20 times, restarted Calibre, my PC, and the Kobo even, and the collections never deleted off the Kobo Manually editing the database was the only way that finally got rid of them