[GUI Plugin] Action Chains

[ownedbycats]

Thanks. I didn't think of a Calibre version problem. Working now!

Tooltip seems to be misbehaving a bit, though. Running on the same chain as before:

calibre, version 6.13.0
ERROR: Cannot run chain: Action Chains: conditions for chain (Read) are not met.
tooltip:

[capink]

Have you actually specified a value for a tooltip in the condition dialog?

[ownedbycats]

Here's what I see





If relevant, I also have an additional condition on a 'View' action. The chain seems to work as expected when 1 book is selected, though.

[ownedbycats]

Oh, tooltip is what I needed to enter. I feel silly now.

[bigwoof]

Hi,

I created a chain to fix all the various tags in my library. it has 4 different single field edit templates all with default scope (whatever is selected).

I selected all the books and ran the chain. it crashed with an out of memory error.

Is there a way to get the chains to run book by book over an entire library? I don't mind waiting a few/many hours for it to be done.

Thanks!

Bigwoof

[ownedbycats]

Possible oversight: Copying a blank 'separator' chain creates a chain titled (copy).

[capink]

Originally Posted by ownedbycats

Possible oversight: Copying a blank 'separator' chain creates a chain titled (copy).

Will be fixed in the next release.

[capink]

Originally Posted by bigwoof

Hi,

I created a chain to fix all the various tags in my library. it has 4 different single field edit templates all with default scope (whatever is selected).

I selected all the books and ran the chain. it crashed with an out of memory error.

Is there a way to get the chains to run book by book over an entire library? I don't mind waiting a few/many hours for it to be done.

Thanks!

Bigwoof

Here is a modified version of the Chain Caller action (called Modified Chain Caller) that can call a target chain multiple times to process chunks of books for each iteration.

#!/usr/bin/env python
# ~*~ coding: utf-8 ~*~

from qt.core import (QApplication, Qt, QWidget, QGridLayout, QHBoxLayout, QVBoxLayout,
                      QGroupBox, QComboBox, QCheckBox, QSpinBox)

import time

from calibre import prints
from calibre.constants import DEBUG

from calibre_plugins.action_chains.chains import Chain
import calibre_plugins.action_chains.config as cfg


from calibre_plugins.action_chains.actions.chain_caller import ChainCallerAction


tooltip = '''
This option splits the books into multiple chunks and run the target chain multiple times,
each time processing a chunk of books. The chunk size can be selected in the spin box.
The book ids in each chunk is passed to the target chain in a variabe called _book_ids.
You must adjust the scope of the actions in the target chains using a template scope to
act on this variable.
'''

class ConfigWidget(QWidget):
    def __init__(self, plugin_action, chain_name, chains_config, *args, **kwargs):
        QWidget.__init__(self)
        self.plugin_action = plugin_action
        self.gui = plugin_action.gui
        self.db = self.gui.current_db
        self.chain_name = chain_name
        self.chains_config = chains_config
        self._init_controls()

    def _init_controls(self):

        l = self.l = QVBoxLayout()
        self.setLayout(l)

        chain_groupbox = QGroupBox(_('Select chain'))
        l.addWidget(chain_groupbox)
        chain_groupbox_l = QVBoxLayout()
        chain_groupbox.setLayout(chain_groupbox_l)
        self.chain_combo = QComboBox()
        chain_groupbox_l.addWidget(self.chain_combo)

        opts_groupbox = QGroupBox(_('Options'))
        opts_groupbox_l = QVBoxLayout()
        opts_groupbox.setLayout(opts_groupbox_l)
        l.addWidget(opts_groupbox)

        vars_chk = self.vars_chk = QCheckBox(_('Pass variables from calling chain'))
        vars_chk.setChecked(False)
        opts_groupbox_l.addWidget(vars_chk)

        #all_chain_names = [chain_config['menuText'] for chain_config in cfg.get_chains_config()]
        all_chain_names = [chain_config['menuText'] for chain_config in self.chains_config if chain_config['menuText']]
        try:
            all_chain_names.remove(self.chain_name)
        except:
            pass
        self.chain_combo.addItems(all_chain_names)
        self.chain_combo.setCurrentIndex(-1)

        chunk_group_box = self.chunk_group_box = QGroupBox('Split books into chunks')
        chunk_group_box.setCheckable(True)
        chunk_group_box.setChecked(False)
        chunk_group_box_l = QVBoxLayout()
        chunk_group_box.setLayout(chunk_group_box_l)
        l.addWidget(chunk_group_box)
        chunk_group_box.setToolTip(tooltip)
        self.chunk_spinbox = QSpinBox()
        self.chunk_spinbox.setMaximum(100000)
        self.chunk_spinbox.setMinimum(1)
        self.chunk_spinbox.setSingleStep(1)
        self.chunk_spinbox.setValue(10)
        chunk_group_box_l.addWidget(self.chunk_spinbox)

        delay_group_box = self.delay_group_box = QGroupBox('Delay')
        delay_group_box_l = QVBoxLayout()
        delay_group_box.setLayout(delay_group_box_l)
        l.addWidget(delay_group_box)
        delay_group_box.setToolTip(_('Delay in seconds between calling chunks'))
        self.delay_spinbox = QSpinBox()
        self.delay_spinbox.setMaximum(100000)
        self.delay_spinbox.setMinimum(0)
        self.delay_spinbox.setSingleStep(1)
        self.delay_spinbox.setValue(0)
        delay_group_box_l.addWidget(self.delay_spinbox)

        l.addStretch(1)

        self.setMinimumSize(300,300)

    def load_settings(self, settings):
        if settings:
            idx = self.chain_combo.findText(settings['chain_name'])
            self.chain_combo.setCurrentIndex(idx)
            self.vars_chk.setChecked(settings.get('pass_vars', False))
            self.chunk_group_box.setChecked(settings.get('split_to_chunks', False))
            self.chunk_spinbox.setValue(settings.get('chunk_size', 10))
            self.delay_spinbox.setValue(settings.get('delay', 0))

    def save_settings(self):
        settings = {}
        settings['chain_name'] = self.chain_combo.currentText()
        settings['pass_vars'] = self.vars_chk.isChecked()
        settings['split_to_chunks'] = self.chunk_group_box.isChecked()
        settings['chunk_size'] = self.chunk_spinbox.value()
        settings['delay'] = self.delay_spinbox.value()
        return settings

class ModifiedChainCallerAction(ChainCallerAction):

    name = 'Modified Chain Caller'
    is_experimental = False

    def run(self, gui, settings, chain):
        # Chain variables {
        add_to_vars = {}
        if settings.get('pass_vars'):
            add_to_vars = self.sanitize_chain_vars(chain)
        add_to_vars['_caller'] = chain.chain_name
        # }
        book_ids = chain.scope().get_book_ids()
        if settings.get('split_to_chunks', False):
            chunk_size = settings['chunk_size']
            chunks = [book_ids[x:x+chunk_size] for x in range(0, len(book_ids), chunk_size)]
        else:
            chunks = [book_ids]
        chunks_no = len(chunks)
        delay = settings.get('delay', 0)
        for i, chunk in enumerate(chunks):
            prints('Action Chains: Chain Caller: running chunk no {} of {}'.format(i, chunks_no))
            add_to_vars['_book_ids'] = ','.join([str(x) for x in chunk])
            target_chain_config = cfg.get_chain_config(settings['chain_name'])
            target_chain = Chain(self.plugin_action, target_chain_config, show_progress=False,
                                chain_vars=add_to_vars, chain_caller=True)
            target_chain.run()
            del target_chain
            if delay:
                if DEBUG:
                    prints(f'sleeping for {delay} seconds')
                time.sleep(delay)

    def config_widget(self):
        return ConfigWidget

You should use this Modified Chain Caller to call your target chain as follows:

1. Add the Modified Chain Caller as a custom action using the module editor (Action Chains > manage modules > create module > copy/paste the code above)
2. Create a new chain that consists of a single action > Modified Chain Caller.
3. In the settings of the Modified Chain Caller, choose your target chain from the dropdown menu.
4. In the settings of the Modified Chain Caller, tick the box for splitting books into chunks, and choose the chunk size (default is 10 books)
5. In your target chain, all the actions must have a template scope with the following template:
   

   program:
       globals(_book_ids)

6. Now select the books you want to process, and run the Modified Chain Caller chain.

I am not sure whether this can help you with the out of memory problem, give it a try and see.

[bigwoof]

Originally Posted by capink

Here is a modified version of the Chain Caller action (called Modified Chain Caller) that can call a target chain multiple times to process chunks of books for each iteration.

Thanks! will try it! I tried some other options and could not quite figure out the right python magic to make it work. interestingly, running my chain on a 1500 books ended up with windows 11 (running on a laptop with 16G ram and 32G ramdisk) rebooting with a rare BSOD...

It works fine with up to 200 books.

Bigwoof

[ownedbycats]

Might be worth running a memtest...

[bigwoof]

Originally Posted by capink

[*]Create a new chain that consists of a single action > Modified Chain Caller.

I can't seem to get this part to work. Even after restarting calibre multiple times (also threw in a reboot in there as well), I can't see the modules appearing as selectable actions in the chain action list. I created a few additional modules to help test as well.

I searched all the other action options and I can't find the modules (searched every entry in calibre actions as well).

I'm using action chains version 1.18.5

Bigwoof

[capink]

After creating the module, the action "Modified Chain Caller" should appear in the dropdown list under "User Actions" node. You don't see that?

[bigwoof]

Originally Posted by capink

After creating the module, the action "Modified Chain Caller" should appear in the dropdown list under "User Actions" node. You don't see that?

Hi,

I can't find a "User Actions" node in the drop down list under actions when you add a new row in a new action chain.

Bigwoof

[capink]

OK. I think I figured out what the problem at your end is. So I modified the module to explicitly set is_experimental to False.

Please try again with the new code (updated the previous post).

[bigwoof]

Originally Posted by capink

OK. I think I figured out what the problem at your end is. So I modified the module (copy below) to explicitly set is_experimental to False.

Thanks! That worked! worked fine on a test of 40 books. running it on 2,000 now :-)



EDIT:

2,000 worked as well. slow and steady wins the race!! no significant CPU or memory build up either. Thank you once again. Action chains are fantastic as a way to change things programatically. And this mechanism allows the chains to run on every book in large libraries..

This would be an excellent addition to the list of examples you already have.. :-)

Note: The problem I was trying to solve with my chains was to reduce the set of tags in my books from ~50,000 to under 200. Most of these tags come from ao3 authors going mad and myself not caring previously.. I finally bit the bullet and wrote some chains to extract only things I care about into tags.. The goal is to make it much easier to find things I care about on my various readers (Android and Kobo) as the number of tags, currently, overwhelms the readers on both those devices and the reader software I use (moon+, lithium, native kobo reader) on those devices don't really look at other attributes beyond title, author, series, and tags.

my other solution, before the "process files in chunks" solution provided by capink, was to edit the sqlite db directly using a sqlite admin tool. However, this has its own issues as the metadata is scattered across multiple tables (needing the right set of join commands) and my SQL foo is weak..

Thanks again!

Bigwoof