[GUI Plugin] Action Chains - Page 8

chaley · 12-16-2020, 02:06 PM

@capink: a random thought. I think that you can get a usable form of action branching if you add an option to run an action only if a template returns a value. You would add this option as another column in the chain links table. Doing this would permit creating a chain where actions are run only if they make sense for the current book.

You could simplify things by requiring use of a zero-parameter stored template. That would ensure you only need a name in the table, and as a side effect would make it easier for the user to test the template.

You could go a bit further and add a check for what the template returns (another column?). If the template returns that value then the action is run. If it does not, the action isn't. That would permit using the same stored template in multiple places.

capink · 12-16-2020, 03:49 PM

Quote:

Originally Posted by chaley

@capink: a random thought. I think that you can get a usable form of action branching if you add an option to run an action only if a template returns a value. You would add this option as another column in the chain links table. Doing this would permit creating a chain where actions are run only if they make sense for the current book.

You could simplify things by requiring use of a zero-parameter stored template. That would ensure you only need a name in the table, and as a side effect would make it easier for the user to test the template.

Funny you mention templates as way to implement flow control, because I had the same idea when writing the plugin, but instead of a separate column for the template, make a template based action that can be placed ahead of an action(s) to optionally jump (skip) over it. The idea is that the template returns a number, if it returns 1 the chain goes on to the next action, if it returns 2 it will skip the next action to the one that follows it, ... and so on (a number more than the total actions remaining would stop the chain).

The current code even have an instance variable in the chain_loop called step_length (default is 1), that is used by the chain loop to jump to the next action (and is ofcourse reset to default after each jump). I was not sure whether this would be useful or not so I deferred it.

Quote:

Originally Posted by chaley

You could go a bit further and add a check for what the template returns (another column?). If the template returns that value then the action is run. If it does not, the action isn't. That would permit using the same stored template in multiple places.

This bit about a separate column for a check is not clear to me.

chaley · 12-16-2020, 05:04 PM

Quote:

Originally Posted by capink

This bit about a separate column for a check is not clear to me.

The idea is to be able to express the following pseudo-code:

Code:

if template() == 'a' then do action 1 fi;
if template() == 'b' then do action 2 fi;
if template() == 'c' then do action 3 fi;

Your idea of having an "action" that can force skipping subsequent actions is very interesting. However, maintenance is an issue, as is knowing the number of actions to skip. That said, your idea leads to what I think is much better than what I proposed, something like this again in pseudo-code.

Code:

v = template_action('foo()')
if v == X then do action 1 fi
if v == X then do action 2 fi
if v == Y then do action 3 fi

Using a template_action would set 'v', a "global variable" in action chains. Subsequent actions would be conditioned by the value of 'v', specified in a column. This way the template returns a value that controls subsequent actions without worrying about changes of state. Any new template_action() would reset that value, permitting expressions like

Code:

v = template1_action('foo()')
if v == X then do action 1 fi
if v == X then do action 2 fi
if v == Y then do action 3 fi
v = template1_action('bar()')
if v == Z then do action 4 fi
if v == W then do action 5 fi

FWIW: if there are additions to the template language that would help, let me know. The main rules are:

A template function cannot change metadata.
There are no globals, which means that a template cannot know something like "what is the largest series number for this series". Of course user defined template functions can do this, but I think that is out of scope.
A template cannot reference more than one book.

compurandom · 12-16-2020, 09:29 PM

While being able to branch would be very powerful, and calling different chains from the end of the branching chain would work and is something I might use, I'd still rather be able to feed a calculated constant down a chain.

I'm thinking something along the lines of maybe code in a module that sets a variable in a special way, and then be able to use that variable in a template where a constant would go in a form in the chain (like, a tag field for instance).

capink · 12-16-2020, 11:01 PM

Quote:

Originally Posted by chaley

Using a template_action would set 'v', a "global variable" in action chains. Subsequent actions would be conditioned by the value of 'v', specified in a column. This way the template returns a value that controls subsequent actions without worrying about changes of state. Any new template_action() would reset that value, permitting expressions like

An action that resets a chain variable, and compared to per action values stored in an extra column, is much better than the jump idea, as it allows finer control over actions. It also presents a visual representation of what action will run under which conditions, and as you said, provides better maintenance when adding/moving actions. Only drawback is that it can be a bit confusing when trying to document/instruct the users on how to use it.

Quote:

Originally Posted by chaley

A template cannot reference more than one book.

Actually I was thinking of not (just) using templates on book metadata, since mi objects can accept extra information I can do something like this:

Code:

device_uuid = get_device_uuid()
cmeta1 = {'datatype': 'text', 'is_multiple': {}, 'name': '#device_uuid'}
mi.set_user_metadata('#device_uuid', cmeta1)
mi.set('#device_uuid', device_uuid)

Using this method, we can insert multiple fields storing global state values enabling the user can define conditions using the familiar template language:

Code:

{#device_uuid}

And now the user can flag actions with the device_uuid (or alternatively device_name if there is such a thing?) and they will only run if the device is attached. This also presents the question of what extra information to inject into the mi object that can be useful for conditional actions?

By the way, the above technique is already used in the plugin. Any action that implements an update_metadata(self, mi) method, is called on every mi object and given the chance to modify it. I have a custom action that inserts a calculated file path, that is not stored in calibre, into the mi object as a #path_to_file field, this field is used by the template feature of the open with action to open the file/folder. Since the file(s) location changes from time to time, I have it mapped into a automatically updated json file (uuid to location mapping).

Finally, when fetching mi object in case we have multiple books selected, which one do we use? the current index? would the metadata for that one book provide value for conditional execution of actions?

@compurandom: See whether the above point about update_metadata() method satisfies your needs, also the code snippet above it on how to insert arbitrary fields into mi objects

Thasaidon · 12-17-2020, 02:04 AM

Hi

I spotted the plugin so thought I would have a look at it.

I downloaded the file but it will not install. I get the following error. I have Windows 10 Enterprise, Calibre 5.4.

Spoiler:

chaley · 12-17-2020, 06:43 AM

Quote:

Originally Posted by capink

Actually I was thinking of not (just) using templates on book metadata, since mi objects can accept extra information I can do something like this:

Code:

device_uuid = get_device_uuid()
cmeta1 = {'datatype': 'text', 'is_multiple': {}, 'name': '#device_uuid'}
mi.set_user_metadata('#device_uuid', cmeta1)
mi.set('#device_uuid', device_uuid)

Using this method, we can insert multiple fields storing global state values enabling the user can define conditions using the familiar template language:

Code:

{#device_uuid}

This is a bit dangerous. You must ensure that the field doesn't already exist. There are also a lot more field metadata items that are assumed to exist in various places, for example "display". Depending on the calibre action, the field might appear in a column list but not work correctly. I don't know if it is safe to pass such an mi to all db functions that modify the database. Note: some formatter functions check if mi is a ProxyMetadata object and refuse to work if it is not. For that reason, and also for performance, you probably want to use get_proxy_metadata() instead of get_metadata().

Another safer way to insert data: implement "globals" in the formatter. I would extend the formatter to accept a dictionary of name:value pairs. A template would access this dict using a new function, perhaps globals(), that acts similarly to the existing arguments() function, copying the globals to local variables. Example: in your plugin you call the formatter with something like

Code:

dict = {'a': 2, 'b': 'some string', 'e':'not used'}

The user puts something like this in the template

Code:

globals(a, b='no string', c='default')

"globals()" sets the variables a, b, and c from the dict, using the default expression if the variable is missing from the dict. In this example 'a' and 'b' are set from the dict, 'c' gets the default, and 'e' isn't copied. NB: template variables can be accessed in both TPM and GPM.

In addition to the above, you define a new built-in action to set the value of a global. It would be almost identical to "Single Field Edit". That along with the branching idea might solve @compurandom's problem because he could use branching to set variables he can use in templates. In fact, he might not need branching.

There might be a way to combine the globals dict with branching. For example, branching could test the global variable '_branch_condition_'. Another way would be to use two columns, one for the global name and one for the value.

Quote:

And now the user can flag actions with the device_uuid (or alternatively device_name if there is such a thing?) and they will only run if the device is attached. This also presents the question of what extra information to inject into the mi object that can be useful for conditional actions?

There is already a formatter function to retrieve the device name, connected_device_name(). I can add a function to get the device UUID if that is something useful.

Regarding extra information, perhaps things like the chain name, the action name, the action comment, the value controlling branching, and action-specific data such as the file path you mention below. FWIW: I think automatically-added globals should have a distinctive name to avoid collisions, perhaps one that begins and ends with '_'.

Quote:

By the way, the above technique is already used in the plugin. Any action that implements an update_metadata(self, mi) method, is called on every mi object and given the chance to modify it. I have a custom action that inserts a calculated file path, that is not stored in calibre, into the mi object as a #path_to_file field, this field is used by the template feature of the open with action to open the file/folder. Since the file(s) location changes from time to time, I have it mapped into a automatically updated json file (uuid to location mapping).

This is useful for modifying metadata. I am still concerned about inserting new metadata fields (lookup keys).

Quote:

Finally, when fetching mi object in case we have multiple books selected, which one do we use? the current index? would the metadata for that one book provide value for conditional execution of actions?

This is an interesting question. There is no reason to believe that the condition evaluation template will generate the same answer for each book. I suspect the right thing to do is (optionally) loop over the books calling the chain for each book. If the user knows that the condition(s) will apply to all books then don't check the option, in which case the condition evaluation uses the first book in the selection.

capink · 12-17-2020, 10:53 AM

Quote:

Originally Posted by Thasaidon

Hi

I spotted the plugin so thought I would have a look at it.

I downloaded the file but it will not install. I get the following error. I have Windows 10 Enterprise, Calibre 5.4.

Spoiler:

Try the new attachment, hopefully it solves the issue.

capink · 12-17-2020, 11:05 AM

Quote:

Originally Posted by chaley

Another safer way to insert data: implement "globals" in the formatter

Again, what you are offering here is much cleaner way of doing things, which would make me do away with the whole update_metadata thing

.

Just to be sure we are on the same page, I will define a global dict for each chain, which have some predefined values and also allows the user to insert his own arbitrary variables

Code:

self.global = {
    _chain_name: ....,
    _action_name: ....,
    user_var1: ......,
    ......
}

Some of the above variables will be overwritten for each action. And this global dict will be used for condition testing as well.

The user can access this using a template like this:

Code:

program: globals('user_var1')

Whenever a template is being evaluated, I should pass this dict to the formatter, may something like this:

Code:

template_output = SafeFormat().safe_format(template, mi, TEMPLATE_ERROR, mi, chain_loop.global)

When testing for conditions, if an action does not define any condition it will always execute. For actions that define conditions, they must be evaluated against the value in the global dict, and if they match, the action will run, otherwise it will be skipped.

One problem here is that since this is a name/value pair, to test conditions you should have two corresponding columns, one for name and another for value. I much prefer that it is one column whose value is tested against a predefined name in the global dict e.g. _condition. Adding multiple columns is confusing, one column with keywords is more accessible IMO. I am open to feedback from others on this.

Edit: striked out since it would not work with one column.

Quote:

Originally Posted by chaley

Note: some formatter functions check if mi is a ProxyMetadata object and refuse to work if it is not. For that reason, and also for performance, you probably want to use get_proxy_metadata() instead of get_metadata().

I didn't know about this. I will modify it for the next release.

Quote:

Originally Posted by chaley

It would be almost identical to "Single Field Edit"

I think a table widget with two columns is more suitable since number and names of variables are not preset. Somewhat similar to table widgets used for chains and actions, with ability to add/remove rows (except for protected name like _chain_name).

Quote:

Originally Posted by chaley

There is already a formatter function to retrieve the device name, connected_device_name(). I can add a function to get the device UUID if that is something useful.

I never use devices with calibre (non rootable kindle, only email one book at a time since collections are useless), so my knowledge here is very limited. I only went for device_uuid as way to prevent name clashes if the user has two similar devices. I will defer to others decide whether a device_uuid is needed.

chaley · 12-17-2020, 12:01 PM

Quote:

Originally Posted by capink

Just to be sure we are on the same page, I will define a global dict for each chain, which have some predefined values and also allows the user to insert his own arbitrary variables

Code:

self.global = {
    _chain_name: ....,
    _action_name: ....,
    user_var1: ......,
    ......
}

Some of the above variables will be overwritten for each action. And this global dict will be used for condition testing as well.

Yes

Quote:

The user can access this using a template like this:

Code:

program: globals('user_var1')

Not quite. The variable name parameters are not quoted, as in globals(user_var1). This defines a template local variable named user_var1.

Note that

Code:

program: globals(user_var1='some_default')

is also valid, as is this TPM template

Code:

{somefield:'globals(user_var1);'rest of template''}

Quote:

Whenever a template is being evaluated, I should pass this dict to the formatter, may something like this:

Code:

template_output = SafeFormat().safe_format(template, mi, TEMPLATE_ERROR, mi, chain_loop.global)

I went ahead and implemented globals in the formatter. The safe_format parameter signature is now:

Code:

    def safe_format(self, fmt, kwargs, error_value, book,
                    column_name=None, template_cache=None,
                    strip_results=True, template_functions=None,
                    global_vars={}):

Using your example you would call it like this:

Code:

template_output = SafeFormat().safe_format(template, mi, TEMPLATE_ERROR,
                                           mi, global_vars=chain_loop.global)

Quote:

When testing for conditions, if an action does not define any condition it will always execute. For actions that define conditions, they must be evaluated against the value in the global dict, and if they match, the action will run, otherwise it will be skipped.

Sounds good. One question: what if the value isn't in the dict? Is that an error or no match?

If you run from source, attached is a version of calibre.utils.formatter.py that implements globals.

For fun, it also implements for loops. Example:

Code:

for v in expression:
    statement [; statement]*
rof

The expression can return a field name, for example 'tags', or it can return a comma-separated list of values. The variable v is set to a value then the statement list is executed. Of course, 'v' can be any variable name.

EDIT: I will submit the formatter changes to Kovid if you think they work for you.

EDIT 2: Replaced the attachment. I had left a print statement in the original.

EDIT 3: Removed the attachment now that the files are in the official calibre source.

capink · 12-17-2020, 01:07 PM

Quote:

Originally Posted by chaley

I went ahead and implemented globals in the formatter. The safe_format parameter signature is now:

Thank You

. I will test tonight as soon I have the time. First mission is to remove the update_metadata and replace with template globals.

Quote:

Originally Posted by chaley

Sounds good. One question: what if the value isn't in the dict? Is that an error or no match?

I haven't decided on that. One option I am evaluating is to defer this to the user. Make clicking on the condition column pop up a template dialog, the user defines his own logic and tests, the template returns a true/false which should guide the plugin on whether to execute the action. The column should display an ellided template with a reasonable width. If the user wants a clue to what the template does, he should used a stored template e.g program: my_kobo_device(), this should be visible in its entirety as clue to when this action runs.

Quote:

Originally Posted by chaley

For fun, it also implements for loops. Example:

That is something I wondered about. Will certainly useful for a lot people

.

ownedbycats · 12-17-2020, 02:50 PM

Small glitch: The formatting appears a bit strange in the plugin updater.

Click image for larger version

Name: 2020-12-17 15_50_26-User plugins.png
Views: 4413
Size: 25.9 KB
ID: 184085

capink · 12-17-2020, 02:58 PM

OK, I am testing this now using calibre-debug -e script.py

Code:

def test():
    from calibre.library import db as DB
    from calibre.ebooks.metadata.book.formatter import SafeFormat
    db = DB().new_api
    book_id = list(db.all_book_ids())[0]
    mi = db.get_proxy_metadata(book_id)
    TEMPLATE_ERROR = 'TEMPLATE_ERROR: '
    my_globals = {'path_to_file': '/home/user', 'b': 2}
    template = "program: globals(path_to_file)"
    template_output = SafeFormat().safe_format(template, mi, TEMPLATE_ERROR, mi, global_vars=my_globals)
    print('debug: template_output: ({})'.format(template_output))
        

test()

the output is empty:

Code:

debug: template_output: ()

I don't know what I am doing wrong?

chaley · 12-17-2020, 03:01 PM

Quote:

Originally Posted by capink

I don't know what I am doing wrong?

Your template "program: globals(path_to_file)" didn't do anything with the variable it set. Try "program: globals(path_to_file); path_to_file", which will return the value of the variable (I hope).

EDIT: I suppose I could have "globals" return the last variable it set instead of the empty string but I am not convinced that is a good idea.

capink · 12-17-2020, 03:15 PM

Thanks, it is working now. It works for me either way. It is your call.

12-17-2020, 02:04 AM	#111
Thasaidon Hedge Wizard Posts: 802 Karma: 19999999 Join Date: May 2011 Location: UK/Philippines Device: Kobo Touch, Nook Simple	Problem Installing Plugin Hi I spotted the plugin so thought I would have a look at it. I downloaded the file but it will not install. I get the following error. I have Windows 10 Enterprise, Calibre 5.4. Spoiler: calibre 5.4 [64bit] embedded-python: True is64bit: True Windows-10-10.0.17763 Windows ('64bit', 'WindowsPE') ('Windows', '10', '10.0.17763') Python 3.8.5 Windows: ('10', '10.0.17763', '', 'Multiprocessor Free') Interface language: None Successfully initialized third party plugins: Comments Cleaner (1, 3, 1) && Diaps Editing Toolbag (0, 3, 7) && EpubCheck (0, 2, 3) && EpubMerge (2, 11, 0) && EpubSplit (3, 0, 0) && Fantastic Fiction (1, 4, 0) && Fantastic Fiction Adults (1, 2, 0) && Favourites Menu (1, 0, 5) && Find Duplicates (1, 8, 3) && Generate Cover (2, 0, 1) && Job Spy (1, 0, 188) && Modify ePub (1, 6, 1) && Open With (1, 5, 13) && Quality Check (1, 11, 0) && Resize Cover (1, 1, 0) && ScrambleEbook (0, 4, 5) && Search The Internet (1, 8, 0) Traceback (most recent call last): File "calibre\gui2\preferences\plugins.py", line 317, in add_plugin File "calibre\customize\ui.py", line 472, in add_plugin File "calibre\customize\ui.py", line 61, in load_plugin File "calibre\customize\zipplugin.py", line 285, in load File "importlib\__init__.py", line 127, in import_module File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "calibre\customize\zipplugin.py", line 177, in exec_module File "calibre\customize\zipplugin.py", line 173, in get_code File "calibre_plugins.action_chains.__init__", line 25 "You can choose from builtin actions, Calibre interface actions, or custom actions " ^ IndentationError: unexpected indent

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Action Chains Resources	capink	Plugins	80	09-18-2025 04:45 AM
[Editor Plugin] Editor Chains	capink	Plugins	106	06-17-2025 05:36 PM
[GUI Plugin] Noosfere_util, a companion plugin to noosfere DB	lrpirlet	Plugins	2	08-18-2022 03:15 PM
[GUI Plugin] Save Virtual Libraries To Column (GUI)	chaley	Plugins	14	04-04-2021 05:25 AM

12-16-2020, 02:06 PM	#106
chaley Grand Sorcerer Posts: 12,525 Karma: 8065948 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	@capink: a random thought. I think that you can get a usable form of action branching if you add an option to run an action only if a template returns a value. You would add this option as another column in the chain links table. Doing this would permit creating a chain where actions are run only if they make sense for the current book. You could simplify things by requiring use of a zero-parameter stored template. That would ensure you only need a name in the table, and as a side effect would make it easier for the user to test the template. You could go a bit further and add a check for what the template returns (another column?). If the template returns that value then the action is run. If it does not, the action isn't. That would permit using the same stored template in multiple places.

12-16-2020, 09:29 PM	#109
compurandom Wizard Posts: 1,026 Karma: 500000 Join Date: Jun 2015 Device: Rocketbook, kobo aura h2o, kobo forma, kobo libra color	While being able to branch would be very powerful, and calling different chains from the end of the branching chain would work and is something I might use, I'd still rather be able to feed a calculated constant down a chain. I'm thinking something along the lines of maybe code in a module that sets a variable in a special way, and then be able to use that variable in a template where a constant would go in a form in the chain (like, a tag field for instance).

12-17-2020, 02:50 PM	#117
ownedbycats Custom User Title Posts: 11,351 Karma: 79528341 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	Small glitch: The formatting appears a bit strange in the plugin updater.

12-17-2020, 03:15 PM	#120
capink Wizard Posts: 1,216 Karma: 1995558 Join Date: Aug 2015 Device: Kindle	Thanks, it is working now. It works for me either way. It is your call.