[GUI Plugin] Action Chains

[davidfor]

Originally Posted by capink

I am not familiar with the count pages plugin, so I cannot give a definitive answer to this.

All I can say, since those changes are going to happen on different actions, there is no risk of corruption, unless count pages starts another thread. And even in the event of a separate threads, calibre is thread safe to the best of my understanding.

Count pages should be safe. It does run in a job, so if something else updated the same columns, while it was running, you would lose that change.

Also, Count Pages supports being called directly from plugins with support for choosing which statistics to calculate and a callback function for the results. I don't remember if there is an option to suppress the prompt. It's been a while since I needed to look at it. I added this at the request of another developer for a private plugin they had.

[ownedbycats]

Originally Posted by davidfor

I don't remember if there is an option to suppress the prompt. It's been a while since I needed to look at it.

[davidfor]

Originally Posted by ownedbycats

Attachment 186244

I meant when called from another plugin. The call can specify some options, but, not all. It doesn't look like suppressing the prompt was one of them..

[ownedbycats]

When Count Pages runs from FanFicFare, it seems to follow the prompt in the settings.

[capink]

Originally Posted by chaley

@capink: Random thought -- does it make sense to give Single Field Edit the option to run over something other than the selection? I was thinking three options:

1. Selection (what it does today).
2. All books, ignoring the selection. What to do about VLs? Either use the current VL or ignore them?
3. Books with ID in book_vars. This one I am not sure of. The idea is that a chain variables action does some computation and sets a book var for what it wants to change. Single field edit would loop over the books with any var set, calling the edit template.

The second is good for actions that want to do multiple things on many books. The third would be convenient for actions that want to do searches then compute results. For example, the second would solve @ownedbycats problem of modifying identifiers without selections. With the third my sample "average rating" function would become two actions with no selection modifiers.

What do you think?

@chaley: Attached to this post is a beta that adds scope management to the plugin. I will leave it for few days for testing before releasing it. I'd like to hear your feedback on this.

Here is how this works:

• By default actions work on books selected in the library view. Scopes gives the user a way to change the scope to include books different to those selected in library view.
• For actions that support scopes, the user will click a button to set the scope for the action (similar to the conditions button). The user will first have to choose a scope manager from the a combobox, after which the settings for chosen scope manager will appear so that the user can configure them.
• The plugin currently ships with one scope manager called "Basic Scopes". It has four options: selected books, all books, books in Book Vars dict, books from search.
• Multiple scope managers can be added including user defined scope managers. More on this below.

Under the hood this is how scopes work:

• Actions that support scopes should set the following attribute:
  

  support_scopes = True

• Whenever and action that have the above attribute set to True wants to get a list of book_ids to work on, it should call the following:
  

  chain.scope().get_book_ids()

  The above will look for scope settings configured by the user for the action, if it finds them, it hands them to the scope manager selected by the user to take care of getting the list of book_ids. If no scope settings are found, it will use a fallback scope manager, which returns a list of books selected in library view.

The whole thing seems to be working well. The only thing that I have doubts about the is user interface for this.

Finally, Here is an example of a custom scope manager that uses a template defined by the user to return the list of book_ids (copy/paste into the module editor):

from __future__ import (unicode_literals, division, absolute_import,
                        print_function)

# python 3 compatibility
from six import text_type as unicode

from PyQt5.Qt import (QApplication, Qt, QVBoxLayout, QGroupBox, QCheckBox, QRadioButton)

from calibre import prints
from calibre.constants import DEBUG
from calibre.utils.search_query_parser import ParseException

from calibre_plugins.action_chains.scopes.base import ActionScope
from calibre_plugins.action_chains.templates import TemplateBox, check_template, get_metadata_object

class TemplateScopeConfigWidget(TemplateBox):

    def __init__(self, plugin_action):
        self.plugin_action = plugin_action
        self.gui = plugin_action.gui
        self.db = self.gui.current_db
        placeholder_text = _('The template entered here should return a comma separated list '
                             'of book_ids')
        TemplateBox.__init__(
            self,
            self.gui,
            plugin_action,
            template_text='',
            placeholder_text=placeholder_text,
            dialog_is_st_editor=True
        )
        vl_chk = self.vl_chk = QCheckBox(_('Exclude books not in current virtual libarary'))
        vl_chk.setChecked(False)
        self.user_layout_1.addWidget(vl_chk)
        
        options_groupbox = QGroupBox(_('Options'))
        self.user_layout_2.addWidget(options_groupbox)
        options_groupbox_layout = QVBoxLayout()
        options_groupbox.setLayout(options_groupbox_layout)
        ids_opt = self.ids_opt = QRadioButton(_('Template output is a list of book ids'))
        options_groupbox_layout.addWidget(ids_opt)
        ids_opt.setChecked(True)
        search_opt = self.search_opt = QRadioButton(_('Template output is a calibre search'))
        options_groupbox_layout.addWidget(search_opt)

    def load_settings(self, settings):
        if settings:
            template = settings['template']
            self.textbox.insertPlainText(template)
            self.vl_chk.setChecked(settings.get('vl'))
            if settings.get('opt') == 'search':
                self.search_opt.setChecked(True)

    def save_settings(self):
        settings = {}
        settings['template'] = unicode(self.textbox.toPlainText()).rstrip()
        settings['vl'] = self.vl_chk.isChecked()
        if self.ids_opt.isChecked():
            settings['opt'] = 'ids'
        elif self.search_opt.isChecked():
            settings['opt'] = 'search'
        return settings


class TemplateScope(ActionScope):

    name = 'Template Scopes'

    def get_book_ids(self, gui, settings, chain):
        db = gui.current_db
        template = settings.get('template', '')
        template_output = chain.evaluate_template(template, book_id=None)
        if settings.get('opt') == 'ids':
            try:
                book_ids = [int(x.strip()) for x in template_output.split(',')]
            except:
                book_ids = []
        elif settings.get('opt') == 'search':
            try:
                book_ids = db.data.search_getting_ids(template_output, '', use_virtual_library=False)
            except ParseException:
                book_ids = []
        if settings.get('vl'):
            vl_ids = self.get_current_restriction_book_ids()
            book_ids = list(set(vl_ids).intersection(set(book_ids)))
        return book_ids

    def validate(self, settings):
        gui = self.plugin_action.gui
        db = gui.current_db
        if not settings:
            return (_('Scope errors'), _('You must configure this scope before running it'))
        mi = get_metadata_object(gui)
        is_template_valid = check_template(settings['template'], self.plugin_action, print_error=False)
        if is_template_valid is not True:
            return is_template_valid
        return True

    def config_widget(self):
        return TemplateScopeConfigWidget

Edit1: Builtin actions that support scopes are:

• Single Field Edit.
• Search And Replace.
• Chain Variables (partial support only for variables that have iterate check enabled).
• Open With (partial support only for multiple selection mode).
• Copy To Clipboard (partial support only for multiple selection mode).

Edit2: The custom single field edit action posted previously will no longer work with this new version.

Edit3: Also the third option in Basic Scopes: "Act on books in that have values in Book Vars" is not at all user friendly and not descriptive enough. If anyone have better alternative?

[Terisa de morgan]

Originally Posted by davidfor

I meant when called from another plugin. The call can specify some options, but, not all. It doesn't look like suppressing the prompt was one of them..

It works. I call CountPages from one plugin and it doesn't ask for updating the column.

[davidfor]

Originally Posted by ownedbycats

When Count Pages runs from FanFicFare, it seems to follow the prompt in the settings.

I do not think that FanFicFare is running Count Pages using the API method that I mentioned. It is should be running it pretty much in the same way as clicking the Count Pages toolbar button. Which means it uses all the configured options of the plugin. The method I am talking about lets the caller pass in options. It means that that you could have the plugin configured to do all of the statistics, but, the caller could just to one of them. Or download the statistics instead of using the book.

But, I did look. For the confirmation, it will do whatever you have configured. But, I'm surprised that I didn't have an option for that. The other option missing is to calculate the various statistics, return them but not update the columns. It wasn't something requested at the time, but, I could see a use for this.

[compurandom]

I run multiple count pages on small batches of books all the time. Very occasionally I get race condition where the count pages initial dialog box gets lost and won't go away, but that's the only issue I've seen. This problem might be worse if you run a separate count pages for each book, all in parallel.

I'd be tempted to select all the uncounted books and run a single count pages job, except that occasionally some large book gets stuck and never finishes, and then all the counts in that job before it are lost.

[chaley]

Originally Posted by capink

@chaley: Attached to this post is a beta that adds scope management to the plugin. I will leave it for few days for testing before releasing it. I'd like to hear your feedback on this.

First comments:

• The "Chain Variables" action template
  

  program:
  	set_book_vars('sum', sum)

  fails with the exception
  

  Traceback (most recent call last):
    File "...\calibre_dev\src\calibre\utils\formatter.py", line 1098, in expr
    File "...\calibre_dev\src\calibre\utils\formatter.py", line 843, in do_node_func
    File "...\calibre_dev\src\calibre\utils\formatter_functions.py", line 136, in eval_
    File "calibre_plugins.action_chains.templates", line 181, in evaluate
  AttributeError: 'ChainInterfaceAction' object has no attribute 'book_vars'

  The same thing happens with "Formula" actions.
• You have a self.resize(700, 750) on line 424 of templates.py. The template dialog now remembers its geometry, so I think you can remove that.
• I don't think you need the big "Icons keys" bar across the middle of the Add actions dialog. Use tooltips on the icons in the table.
• You should disable the "Action scopes" combo box if the action doesn't support scopes. More about this below.
• I would consider getting rid of the new first-level button dialog, replacing it with top-level menu items. As it is that dialog forces "extra" clicks. The only one missing from the top-level menu is "Keyboard Shortcuts. Reserve "Customize plugin" for when the user is doing something plugin-wide, such as if you add fonts or color schemes. Alternatively use a second-level menu.
• Perhaps put tooltips on the menu items that "configure", adding more information about what they do.

I had trouble working out what buttons did and when I should/could use them. The following would be easier for me, but perhaps not everyone. The suggestions are predicated on using two columns for the icons, one for the conditions and one for the scopes. Visually their vertical alignment is "better", it avoids the centering problem, and it makes a blank space significant.

I would then for action conditions:

• Add a "lock open" icon. This icon shows when there isn't a condition.
• Double-clicking on either the open or closed lock brings up the condition dialog.
• Right-clicking on the closed lock brings up a context menu offering "edit" and "delete"
• Right-clicking on an open icon offers "create", which is the same as edit.
• If it is possible to have actions on which a condition cannot be applied, put no icon in the cell.
• Remove the "Conditions" combo box.

For scopes:

• Don't have an "Action doesn't support scopes" icon. An empty table cell would indicate that no scope can be set. If you want to be fancy you could add a tooltip to the empty table cell saying that no scope is possible
• Repurpose the "crossed eye" as "scope not set"
• Double-clicking on the "scope set" icon (the uncrossed eye) opens the selected scope manager.
• Right-clicking on the "scope set" icon opens a context menu offering edit and delete scope. Hmmm... perhaps better words are "attach scope" and "detach scope".
• Double-clicking on the "scope not set" icon brings up the scope selector.
• Right-clicking on the scope not set icon brings up a context menu offering the scope selector. Same as double-clicking.
• Remove the "Scopes" combo box.

The scoping stuff works as I expect. I changed my sample "Average Ratings" chain to use the scope "Act on all books", deleting the selection modifiers. It worked, was fast, and didn't mess with the selection in the library view.

I couldn't test some things because of the chain_vars exception.

Regarding:

Finally, Here is an example of a custom scope manager that uses a template defined by the user to return the list of book_ids (copy/paste into the module editor):

This sounds like a fine standard scope manager.

Regarding

Edit3: Also the third option in Basic Scopes: "Act on books in that have values in Book Vars" is not at all user friendly and not descriptive enough. If anyone have better alternative?

You might consider short labels with long tooltips that explain what it means. I think it is impossible to have labels that both are concise and explain what the user needs to know. Tooltips are used as documentation throughout calibre so a calibre user would expect that.

[capink]

Originally Posted by chaley

[LIST][*]The "Chain Variables" action template

program:
	set_book_vars('sum', sum)

fails with the exception

Traceback (most recent call last):
  File "...\calibre_dev\src\calibre\utils\formatter.py", line 1098, in expr
  File "...\calibre_dev\src\calibre\utils\formatter.py", line 843, in do_node_func
  File "...\calibre_dev\src\calibre\utils\formatter_functions.py", line 136, in eval_
  File "calibre_plugins.action_chains.templates", line 181, in evaluate
AttributeError: 'ChainInterfaceAction' object has no attribute 'book_vars'

The same thing happens with "Formula" actions.

This is not happening for me. I need the exported chain zip to trigger the error.

Originally Posted by chaley

You have a self.resize(700, 750) on line 424 of templates.py. The template dialog now remembers its geometry, so I think you can remove that.

It does not remember the size for me (calibre 5.14) e.g. formulas dialog. If I remove it scrollbars kick in for both directions. I have to resize manually everytime I use the dialog. I just tested again by commenting out the resizing, same result.

Originally Posted by chaley

I don't think you need the big "Icons keys" bar across the middle of the Add actions dialog. Use tooltips on the icons in the table.

I will remove the bar. Tooltips are already used.

Originally Posted by chaley

I would consider getting rid of the new first-level button dialog, replacing it with top-level menu items. As it is that dialog forces "extra" clicks. The only one missing from the top-level menu is "Keyboard Shortcuts. Reserve "Customize plugin" for when the user is doing something plugin-wide, such as if you add fonts or color schemes. Alternatively use a second-level menu.

As you already pointed all items (except keyboard shortcuts) already have top-level menu items, so no need for extra clicks. The chains dialog now have a separate menu item called "Add/Modify chains".

The button dialog was added not for the buttons, but for setting global options for the plugin. I had one in a transitional version that I removed, but I decided to keep the dialog for future use. If this is confusing, I can hide it until I need it.

Originally Posted by chaley

The suggestions are predicated on using two columns for the icons, one for the conditions and one for the scopes.

I intially started this way but I didn't like it and scrapped it. The logic here is I don't want to clutter the table with extra columns, which most probably will not be used by most people (conditions and scopes), as it might add to the confusion. Maybe more people weighing on this would help resolve this issue.



I like your idea of clikcing on icons triggering the appropriate dialogs, but there are two problems with this:

• As I explained previously, I am not sure about the two column solution. And with one icons column, I do not know how to intercept a click on one of the icons in a cell. I presume this is possible and will look on how to do it. But I am not sure how far I can go with this.
• The dropdown button while not a pretty solution, has the advantage of allowing setting the same condition/scope for multiple actions at once. This can be replaced by context menu. The current beta already have context menu with add and remove scopes items. I can add two items for conditions as well.

Originally Posted by chaley

Don't have an "Action doesn't support scopes" icon. An empty table cell would indicate that no scope can be set

I had this because as I said in the previous paragraph, a user can use the button to set scopes on multiple actions simultaneously, and I wanted to give hints about actions that are not going respond to this. Thought maybe that would be more clear than having a blank cell (while using the negated scope icon for action that support but does not have scopes). I will re-think this.

Originally Posted by chaley

You might consider short labels with long tooltips that explain what it means. I think it is impossible to have labels that both are concise and explain what the user needs to know. Tooltips are used as documentation throughout calibre so a calibre user would expect that.

I will shorten the label. But I am still having trouble with explaining book vars in detail.

[chaley]

Originally Posted by capink

This is not happening for me. I need the exported chain zip to trigger the error.

And now it isn't happening for me. I did reinstall the plugin, which might have fixed something.

It does not remember the size for me (calibre 5.14) e.g. formulas dialog. If I remove it scrollbars kick in for both directions. I have to resize manually everytime I use the dialog. I just tested again by commenting out the resizing, same result.

The reason that the template dialog isn't saving its geometry is that you aren't calling its accept function. Line 436 in "templates.py" should be

            TemplateDialog.accept(self)

instead of

            QDialog.accept(self)

Line 439 in reject() needs a similar change.

[capink]

IIRC TemplateDialog.reject(self) was problematic because it also closes the parent dialog (which is the chains dialog when the template dialog is a conditions dialog). That is why I implemented reject in the first place. I will call self.save_geometry() from both, unless I shouldn't for some reason.

[chaley]

Originally Posted by capink

IIRC TemplateDialog.reject(self) was problematic because it also closes the parent dialog (which is the chains dialog when the template dialog is a conditions dialog). That is why I implemented reject in the first place. I will call self.save_geometry() from both, unless I shouldn't for some reason.

Use TemplateDialog.accept() but not .reject() and don't call save_geometry(). If you don't want to do that then don't call save_geometry() in reject. TemplateDialog itself does not call it in reject().

[capink]

Ok, will call TemplateDialog.accept() only.

[chaley]

Originally Posted by capink

This is not happening for me. I need the exported chain zip to trigger the error.

I had it happen again. It seems to happen only when a chain is being defined. Once the chain is defined and saved it stops. I also restarted calibre in the middle of that but I don't think that changed the behavior.