[GUI Plugin] Action Chains

[capink]

It is safe to modify the chain, but the chain will not run unless that action is available.

If you have a scenario in which you want to run the chain without the device plugged in; you have to set a connected_device_name() condition on that particular action, and also tick the option in the conditions evaluator to not validate the action if the conditions are not met.

[ownedbycats]

I use connected_device_name for one of my chains. If the device is connected, it sends to device; if not it adds it to my "send to device" reading list to run when the device gets connected.

I just wanted to make sure it wouldnt remove 'invalid' actions when i saved it.

[chaley]

@capink: Random thought -- does it make sense to give Single Field Edit the option to run over something other than the selection? I was thinking three options:

1. Selection (what it does today).
2. All books, ignoring the selection. What to do about VLs? Either use the current VL or ignore them?
3. Books with ID in book_vars. This one I am not sure of. The idea is that a chain variables action does some computation and sets a book var for what it wants to change. Single field edit would loop over the books with any var set, calling the edit template.

The second is good for actions that want to do multiple things on many books. The third would be convenient for actions that want to do searches then compute results. For example, the second would solve @ownedbycats problem of modifying identifiers without selections. With the third my sample "average rating" function would become two actions with no selection modifiers.

What do you think?

[capink]

Running over non selected books is something I always pondered, not just for Single Field Edit, but for all builtin actions with the exception of calibre actions, which have to be always selection based, since the actions are not under the plugin control.

The way I imagined it was a template based solution, where the template returns list of ids. For example, to run over all ids the template below can be used:

program:
from_search('id', '')

The book_vars ids will need a new template function, since book_vars always act on one selected row (the row containing the current index). Or maybe modify both book_vars and set_book_vars and make them accept id values.

The stumbling block was whether to implement this for each action separately, or can there be a better way to implement a general mechanism used by all actions. Any suggestions would be welcome. This might take some time until I reach a solution I am comfortable with.

Until then we can experiment with custom actions. To that end, I will introduce a change in the next version which will add a new action method — maybe call it loop_ids() — which can be overridden by subclasses to return the ids that will be acted on by the action.

Edit: One concern with this approach will be how handle invalid template output as they will be difficult to validate ahead of time.

[chaley]

Originally Posted by capink

Running over non selected books is something I always pondered, not just for Single Field Edit, but for all builtin actions with the exception of calibre actions, which have to be always selection based, since the actions are not under the plugin control.

That is a good idea.

The way I imagined it was a template based solution, where the template returns list of ids. For example, to run over all ids the template below can be used:

program:
from_search('id', '')

I thought of this. My concern was that the template functions explode string lists, which is *very slow* for large lists such as thousands of books. If you (we) go down this road then we might want to introduce a specific type and associated functions for book lists that uses a python set underneath. I think this would be OK in the specific case of lists (sets) of numbers. I need to look at this if we want to try it.

The book_vars ids will need a new template function, since book_vars always act on one selected row (the row containing the current index). Or maybe modify both book_vars and set_book_vars and make them accept id values.

My idea was to use the key to the book_vars dict, which as far as I can tell is always the book id (I think book_vars is {book_id: {var_name: value}}). Setting any var would add the book to the list to be processed. The template would need to know what it is doing.

That said, a chain_vars-like action that returns a list of books is probably the best solution. That gets us back to the above questions...

The stumbling block was whether to implement this for each action separately, or can there be a better way to implement a general mechanism used by all actions. Any suggestions would be welcome. This might take some time until I reach a solution I am comfortable with.

I agree. I sometimes don't think enough about solutions and end up making something that doesn't work very well but must be maintained for compatibility.

You (we) also need to ensure that the complexity doesn't rise beyond the "usable point". That is a problem I have had with the template language for the last 10 years.

Edit: One concern with this approach will be how handle invalid template output as they will be difficult to validate ahead of time.

I think that functional validation is hopeless. Who knows what the "template program" is supposed to do, and who knows if it is doing it correctly? Only the user. Verifying syntax is about the best one can do, unless some test mechanism is introduced where the action is run but nothing is permanent. How to do that in a chain of actions escapes me.

[capink]

Originally Posted by chaley

I thought of this. My concern was that the template functions explode string lists, which is *very slow* for large lists such as thousands of books. If you (we) go down this road then we might want to introduce a specific type and associated functions for book lists that uses a python set underneath. I think this would be OK in the specific case of lists (sets) of numbers. I need to look at this if we want to try it.

I made a chain (attached below) to test this by having a template store the ids to chain variable, and have a second custom action read the variable and convert to python list. You have to add this custom action through the module editor before using the attached chain:

from calibre_plugins.action_chains.actions.base import ChainAction

class MyTestAction(ChainAction):

    name = 'My Test Action'

    def run(self, gui, settings, chain):
        book_ids = chain.chain_vars['book_ids']
        book_ids = book_ids.split(',')
        book_ids = [int(book_id) for book_id in book_ids]
        print(book_ids)

It is running blindingly fast (0.3 seconds on a 52000 books library). Much faster than the usual call to calculate book ids from selected rows.

Originally Posted by chaley

My idea was to use the key to the book_vars dict, which as far as I can tell is always the book id (I think book_vars is {book_id: {var_name: value}}).

You are correct. The dictionary keys (book_ids) can be accessed from any action. I was talking about accessing them from a template.

Originally Posted by chaley

You (we) also need to ensure that the complexity doesn't rise beyond the "usable point". That is a problem I have had with the template language for the last 10 years.
I think that functional validation is hopeless. Who knows what the "template program" is supposed to do, and who knows if it is doing it correctly? Only the user. Verifying syntax is about the best one can do, unless some test mechanism is introduced where the action is run but nothing is permanent. How to do that in a chain of actions escapes me.

You are probably right about validation. But still we need to make a decision whether to abort the action or the whole chain.

Edit: modify custom action make it convert book_ids to integer values.

[ownedbycats]

Originally Posted by capink

You are probably right about validation. But still we need to make a decision whether to abort the action or the whole chain.

In general (not just for template issues) I think it would be a good idea to prompt "this action failed, continue rest of chain?" Some chains depend on specific action (or a series of actions) working correctly while others don't.

[chaley]

Originally Posted by ownedbycats

In general (not just for template issues) I think it would be a good idea to prompt "this action failed, continue rest of chain?" Some chains depend on specific action (or a series of actions) working correctly while others don't.

How do you know that it failed? Syntax errors and exceptions are the easy ones.

Perhaps I should offer an error mechanism in the template language so that the template itself can indicate that something went wrong. Something like error('message') that throws a python ValueError.

[ownedbycats]

Yes that really only applies if theres an error.

[capink]

The plugin offers a builtin event called "Book List Double Clicked". Using this event, you can either replace calibre default action when double clicking on books with your own chain(s), or you can use your chain(s) to supplement calibre default action. If you want to completely replace calibre's default action with your chain(s), you have to edit the following tweak first to disable the default calibre action when double clicking (calibre preferences > tweaks):

doubleclick_on_library_view = 'do_nothing'

However, editing the above tweak has the side effect of disabling editing a field using a single click. If you want to keep the ability to edit a field using single click after adding the above tweak, actions chains gives you this option, by adding another tweak (calibre preferences > tweaks > plugin tweaks):

action_chains_single_click_edit = True

Note: Book List Double Clicked event passes a couple of arguments that are stored as a chain variable called _event_args. The two arguments passed are the book_id, and column name.

Note: If the book is double clicked in the Cover Grid, the value for column_name will be an empty string.

You can use this to make a chain that take different courses depending the cell you double clicked. For example; open the book in the viewer when the user clicks on the title, and open the tag editor using this custom action if the user clicks a tag like cell.

For the above to work, you have to set conditions on the actions in the chain, so that each action is only run when the appropriate cell is clicked. Here is an example of a condition that will make the action run only if the a tag like cell is clicked:


Template:

program:
	clicked_column = list_item(globals(_event_args), 1, ',');
	tag_like = field_names('is_multiple:true,is_names:false');
	str_in_list(tag_like, ',', clicked_column, 'true', '')

Datatype: text
Comparison: =
Condition Value: true

[capink]

Version 1.5.0

• Update: Calibre Actions: Add option to wait for jobs started by action to finish before proceeding.
• Update: Calibre Actions: Add option to wait for progress bar started by action to finish before proceeding.
• Update: Calibre Actions: re-add previously exluded action like add books, remove books, convert books ... etc as they can be used with newly introduced options.
• Update: Add new event: Library View Double Cliked.
• Update: New template function to_timestamp which makes it possible to do date comparisons by converting date to float numbers.
• Fix: Calibre Actions: Bug that allowed choosing two actions if they are not top level items.
  
• API changes: Rename some methods.

The biggest change for this version is that Calibre Actions is now able use actions like Add books, convert books ... etc. The reason they were not included is outlined in the quote below:

Originally Posted by capink

Calibre Actions does expose most of calibre interface actions, but not all of them are suitable for use with this plugin. The reason is that some of these actions have their own progress bar, so they run the code on a different thread. So whenever you use an of these actions, the next action in the chain will start before they finish because they are on a different thread.

Some other actions, like convert books, run one or more jobs, so the same problem is happening here. There is no way to run these action sequentially with other actions. They can only be used as the last action in a chain.

I have removed some of these actions like add books, remove books, convert books. But I cannot hunt down every possible action that might cause such problems, so you always have to test before using them.

The two newly introduced options solve these problems. The first one if checked allows the plugin to detect jobs started by selected action and wait until these jobs are finished, and then proceed with the rest of the chain. The second allows the plugin to detect progress bar started by selected action and block the main thread until the progress bar disappears.

However, there is only one remaining limitation; as the detection of a progress bar fails whenever the user check the "Hide Main Window" from the tray icon. So avoid using these together.

The new double click event can be used to launch chain(s) in response to double clicking items on the library view. It might be better to disable the default action associated with double clicking if you want to use this event. (preferences > tweaks > doubleclick_on_library_view > do_nothing)

Also I am not sure if the functionality of to_timestamp() is already present in calibre's template language. I checked format_date() and it does not seem to offer this possibility. If this is not the case, or if you think such functionality fits better into calibre template language, I will remove this function from future versions.

[chaley]

Originally Posted by capink

@chaley: I posted a custom action based on Single Field Edit that can run over books other than those selected. It will only work the latest version.

I tried it using an action that stores the book_id to a column. It worked very nicely. I will try the book_vars variant later.

Also I am not sure if the functionality of to_timestamp() is already present in calibre's template language. I checked format_date() and it does not seem to offer this possibility. If this is not the case, or if you think such functionality fits better into calibre template language, I will remove this function from future versions.

There are currently two ways of doing it.

• days_between(d1, d2) -- returns the number of days between the two dates. Resolution is 1/10th of a day.
  

  days_between($last_modified, $#mydate))

• Comparing date strings:
  

  if format_date($last_modified, 'iso') > format_date($#mydate, 'iso') then

I don't know what you are trying to do so I don't know if these are sufficient.

However, I think a good solution is to add the 'timestamp' format to format_date, returning a floating point timestamp. I would also change format_date() to allow a timestamp as the first argument (the "date") permitting conversion of timestamps to date strings. This would do what you need and also facilitate arithmetic on dates. Example:

program:
	t = format_date($last_modified, 'timestamp');
	strcat(t, ' -- ', format_date(t, 'yyyy MMM ddd hh:MM:ss'))

Result: 1616695611.0 -- 2021 Mar Thu 18:03:51

One possible problem with both implementations: the dates lose their timezone. I don't see that as a big deal.

[chaley]

Originally Posted by chaley

However, I think a good solution is to add the 'timestamp' format to format_date, returning a floating point timestamp. I would also change format_date() to allow a timestamp as the first argument (the "date") permitting conversion of timestamps to date strings. This would do what you need and also facilitate arithmetic on dates. Example:

program:
	t = format_date($last_modified, 'timestamp');
	strcat(t, ' -- ', format_date(t, 'yyyy MMM ddd hh:MM:ss'))

Result: 1616695611.0 -- 2021 Mar Thu 18:03:51

That idea didn't work out. There were too many ambiguities. Instead I added the date format strings:

• to_number : convert the date & time into a floating point number (a timestamp)
• from_number : convert a floating point number (a timestamp) into an iso formatted date. If you want a different date format then add the desired formatting string after from_number and a colon ( : ). Example: from_number:MMM dd yyyy

I used '_number' instead of '_timestamp' because I thought that 'number' better describes what it is.

Example:

program:
	t = format_date(today(), 'to_number');
	format_date(t, 'from_number:MMM dd yyyy')

Result: Mar 25 2021

I submitted the changes to Kovid to see what he thinks.

[capink]

Originally Posted by chaley

I tried it using an action that stores the book_id to a column. It worked very nicely. I will try the book_vars variant later.

Good to know.

Originally Posted by chaley

Comparing date strings:

if format_date($last_modified, 'iso') > format_date($#mydate, 'iso') then

Never thought about comparing iso strings. Nice trick.

Originally Posted by chaley

However, I think a good solution is to add the 'timestamp' format to format_date, returning a floating point timestamp. I would also change format_date() to allow a timestamp as the first argument (the "date") permitting conversion of timestamps to date strings. This would do what you need and also facilitate arithmetic on dates. Example:

program:
	t = format_date($last_modified, 'timestamp');
	strcat(t, ' -- ', format_date(t, 'yyyy MMM ddd hh:MM:ss'))

Result: 1616695611.0 -- 2021 Mar Thu 18:03:51

One possible problem with both implementations: the dates lose their timezone. I don't see that as a big deal.

Sounds like a perfect solution. The timezone thing is not important IMO. I will remove the to_timestamp() from the next version as yours is much better.

[Terisa de morgan]

Originally Posted by capink

Version 1.5.0

• Update: Calibre Actions: Add option to wait for jobs started by action to finish before proceeding.
• Update: Calibre Actions: Add option to wait for progress bar started by action to finish before proceeding.
• Update: Calibre Actions: re-add previously exluded action like add books, remove books, convert books ... etc as they can be used with newly introduced options.
• Update: Add new event: Library View Double Cliked.
• Update: New template function to_timestamp which makes it possible to do date comparisons by converting date to float numbers.
• Fix: Calibre Actions: Bug that allowed choosing two actions if they are not top level items.
  
• API changes: Rename some methods.

The biggest change for this version is that Calibre Actions is now able use actions like Add books, convert books ... etc. The reason they were not included is outlined in the quote below:



The two newly introduced options solve these problems. The first one if checked allows the plugin to detect jobs started by selected action and wait until these jobs are finished, and then proceed with the rest of the chain. The second allows the plugin to detect progress bar started by selected action and block the main thread until the progress bar disappears.

However, there is only one remaining limitation; as the detection of a progress bar fails whenever the user check the "Hide Main Window" from the tray icon. So avoid using these together.

The new double click event can be used to launch chain(s) in response to double clicking items on the library view. It might be better to disable the default action associated with double clicking if you want to use this event. (preferences > tweaks > doubleclick_on_library_view > do_nothing)

@chaley: I posted a custom action based on Single Field Edit that can run over books other than those selected. It will only work the latest version.

Also I am not sure if the functionality of to_timestamp() is already present in calibre's template language. I checked format_date() and it does not seem to offer this possibility. If this is not the case, or if you think such functionality fits better into calibre template language, I will remove this function from future versions.

Thank you ,I think these options can simplify my process.