[Editor Plugin] Editor Chains

[skil-phil]

Originally Posted by capink

I pushed a fix for this. Try and see how it goes.

Thanks works for me as well. Nice addition.

[ownedbycats]

Small question: I made an editor chain that runs a search-and-replace on the internal table of contents (to strip link anchors).

For the filter, has anybody ever seen that file named something other than toc.ncx?

[nqk]

nav.xhtml

Sent from my Pixel 7 Pro using Tapatalk

[DNSB]

Originally Posted by ownedbycats

Small question: I made an editor chain that runs a search-and-replace on the internal table of contents (to strip link anchors).

For the filter, has anybody ever seen that file named something other than toc.ncx?

I've seen it with several names but I've only once seen where it did not have the .ncx extension which as far as I read from the specification should not have been permitted. For ePub3, the navigation document is just another .xhtml file and the name is not important.

I've actually needed to look at the .opf file to double check the filenames on occasion.

[ownedbycats]

Thank you both

[capink]

Originally Posted by DNSB

For ePub3, the navigation document is just another .xhtml file and the name is not important.

I've actually needed to look at the .opf file to double check the filenames on occasion.

Thanks for the info. I was operating under the impression that nav.xhtml is mandatory name. Will address this in the next version by looking for the name in the opf. The plugin already looks only for ncx extension regardless of the name for epub2.

[nqk]

I would like to request for "Download external resources" action.

Thank you for the plugin.

👍👍👍👍

Sent from my Pixel 7 Pro using Tapatalk

[capink]

Originally Posted by nqk

I would like to request for "Download external resources" action.

Thank you for the plugin.

👍👍👍👍

Sent from my Pixel 7 Pro using Tapatalk

Here is a custom action to do what you want. You have to copy/paste the code below into a new module (Editor Chains > Module manager > Add):

import traceback

from calibre.utils.localization import ngettext
from polyglot.builtins import iteritems
from calibre.ebooks.oeb.polish.download import (
    download_external_resources, get_external_resources, replace_resources,
)

from calibre_plugins.editor_chains.actions.base import EditorAction

class ExternalResources(EditorAction):

    name = 'Download External Resources'
    headless = True

    def run(self, chain, settings, *args, **kwargs):
        container = chain.current_container
        try:
            resources = get_external_resources(container)
        except Exception as err:
            print(err, traceback.format_exc())
            return
        if not resources:
            print(_('No external resources were found in this book.'))
            return
        try:
            downloaded = download_external_resources(container, resources)
        except Exception as err:
            print(_('Failed to download external resources.'))
            print(err, traceback.format_exc())
            return

        replacements, failures = downloaded
        if failures:
            tb = [f'{url}\n\t{err}\n' for url, err in iteritems(failures)]
            det_msg='\n'.join(tb)
            if not replacements:
                print(_(
                    'Failed to download external resources.'), det_msg)
                return
            else:
                print(_(
                    'Warning: Failed to download some external resources.'), det_msg)

        try:
            ret = replace_resources(container, resources, replacements)
        except Exception as err:
            print(_('Failed to replace external resources'))
            print(err, traceback.format_exc())
            return

    def validate(self, settings):
        return True

This is not something I am welling to work on debugging, so that is why I am not including it as part of the plugin. If you test it for some time and it is working without problems, maybe it can be added in the future.

[thiago.eec]

Hi, @capink.

Thanks for the great plugin!

I've just created my first chain. It is a 'clean up' chain to correct the most common problems I usually find in my books.
There were only two things that I missed and I was wondering if they could be added.

1) Add the 'Check book' action
2) Call other plugin's action (like running Epubcheck or ACE).

Do you think this would be possible?

[capink]

Originally Posted by thiago.eec

2) Call other plugin's action (like running Epubcheck or ACE).

There is a custom action you can add that would do what you want. Go to Editor Chains > Manage Modules > Create > Copy/Paste the following:

#!/usr/bin/env python
# ~*~ coding: utf-8 ~*~
from __future__ import (unicode_literals, division, absolute_import,
                        print_function)

import re

# python 3 compatibility
from six import text_type as unicode

from PyQt5.Qt import (QApplication, Qt, QWidget, QGridLayout, QHBoxLayout, QVBoxLayout,
                      QDialog, QDialogButtonBox, QMenu, QPainter, QPoint, QPixmap,
                      QSize, QIcon, QTreeWidgetItem, QTreeWidget, QAbstractItemView,
                      QGroupBox, QCheckBox, QLabel, QAction)

from calibre import prints
from calibre.constants import DEBUG
from calibre.gui2 import error_dialog

from calibre_plugins.editor_chains.actions.base import EditorAction
from calibre_plugins.editor_chains.common_utils import get_icon

try:
    load_translations()
except NameError:
    prints("EditorChains::actions/editor_actions.py - exception when loading translations")


EXCLUDED_GROUPS = [
    'Editor Chains',
    'Editor actions',
    'Windows',
    'Preview',
    'Search'
]

ICON_SIZE = 32


def group_unique_names(gui, group):
    menu_actions = [x for x in gui.__dict__.values() if isinstance(x, QAction)]
    menu_unique_names = [re.sub(r'^action\-', r'', x.objectName()) for x in menu_actions]
    if group == 'Plugins':
        unique_names = [x for x in gui.keyboard.groups['Plugins']]
    else:
        unique_names = [x for x in gui.keyboard.groups[group] if x in menu_unique_names]
    
    return unique_names

def get_qaction_from_unique_name(gui, unique_name):
    shortcut = gui.keyboard.shortcuts[unique_name]
    ac = shortcut['action']
    return ac


def allowed_unique_names(gui):
    unique_names = []
    for group in gui.keyboard.groups.keys():
        unique_names.extend(group_unique_names(gui, group))
    unique_names += [x for x in gui.keyboard.groups['Plugins']]
    return unique_names

class Item(QTreeWidgetItem):
    pass

class ConfigWidget(QWidget):
    def __init__(self, plugin_action):
        QWidget.__init__(self)
        self.plugin_action = plugin_action
        self.gui = plugin_action.gui
        self.all_nodes = {}
        # used to keep track of selected item and allow only one selection
        self.checked_item = None
        
        self._init_controls()

    def _init_controls(self):
        self.blank_icon = QIcon(I('blank.png'))
        l = self.l = QVBoxLayout()
        self.setLayout(l)

#        opts_groupbox = QGroupBox()
#        l.addWidget(opts_groupbox)
#        opts_groupbox_layout = QVBoxLayout()
#        opts_groupbox.setLayout(opts_groupbox_layout)
#        cursor_chk = self.cursor_chk = QCheckBox(_('Disable wait cursor for this action.'))
#        opts_groupbox_layout.addWidget(cursor_chk)
#        cursor_chk.setChecked(False)
        
        self.tv = QTreeWidget(self.gui)
        self.tv.setIconSize(QSize(ICON_SIZE, ICON_SIZE))
        self.tv.header().hide()
        self.tv.setSelectionMode(QAbstractItemView.SingleSelection)
        l.addWidget(self.tv, 1)
        self.tv.itemChanged.connect(self._tree_item_changed)

        self._populate_actions_tree()

    def _get_scaled_icon(self, icon):
        if icon.isNull():
            return self.blank_icon
        # We need the icon scaled to 16x16
        src = icon.pixmap(ICON_SIZE, ICON_SIZE)
        if src.width() == ICON_SIZE and src.height() == ICON_SIZE:
            return icon
        # Need a new version of the icon
        pm = QPixmap(ICON_SIZE, ICON_SIZE)
        pm.fill(Qt.transparent)
        p = QPainter(pm)
        p.drawPixmap(QPoint(int((ICON_SIZE - src.width()) / 2), int((ICON_SIZE - src.height()) / 2)), src)
        p.end()
        return QIcon(pm)

    def set_checked_state(self, item, col, state):
        item.setCheckState(col, state)
        if state == Qt.Checked:
            self.checked_item = item
        else:
            self.checked_item = None

    def _populate_actions_tree(self):

        self.top_level_items_map = {}
        for group in sorted(self.gui.keyboard.groups.keys()):
            if group in EXCLUDED_GROUPS: continue
            
            # Create a node for our top level plugin name
            tl = Item()
            tl.setText(0, group)
          
            # Normal top-level checkable plugin iaction handling
            tl.setFlags(Qt.ItemIsEnabled)
            
            #unique_names = self.gui.keyboard.groups[group]
            unique_names = group_unique_names(self.gui, group)
            
            # Iterate through all the children of this node
            self._populate_action_children(unique_names, tl)
            self.tv.addTopLevelItem(tl)

    def _populate_action_children(self, unique_names, parent):
        for unique_name in unique_names:
            ac = get_qaction_from_unique_name(self.gui, unique_name)
            text = ac.text().replace('&', '')

            it = Item(parent)
            it.setText(0, text)
            it.setData(0, Qt.UserRole, unique_name)
            it.setFlags(Qt.ItemIsEnabled | Qt.ItemIsUserCheckable)
            it.setCheckState(0, Qt.Unchecked)
            it.setIcon(0, self._get_scaled_icon(ac.icon()))

            self.all_nodes[unique_name] = it

    def _tree_item_changed(self, item, column):
        # Checkstate has been changed

        is_checked = item.checkState(column) == Qt.Checked
        #text = unicode(item.text(column)).replace('&', '')
        unique_name = item.data(column, Qt.UserRole)

        if is_checked:
            # un-check previously checked item if any
            if self.checked_item:
                self.tv.blockSignals(True)
                self.checked_item.setCheckState(0, Qt.Unchecked)
                self.tv.blockSignals(False)
            
            self.checked_item = item
        else:
            self.checked_item = None

    def _repopulate_actions_tree(self):
        self.tv.clear()
        self._populate_actions_tree()

    def load_settings(self, settings):
        if settings:
            #self.cursor_chk.setChecked(settings.get('disable_busy_cursor', False))
            #self._repopulate_actions_tree()
            self.set_unique_name_checked(settings['unique_name'])

    def save_settings(self):
        settings = {}
        settings['unique_name'] = self.checked_item.data(0, Qt.UserRole)
        #settings['disable_busy_cursor'] = self.cursor_chk.isChecked()
        return settings

    def set_unique_name_checked(self, unique_name):
        it = self.all_nodes.get(unique_name)
        if it:
            self.set_checked_state(it, 0, Qt.Checked)
            self.tv.setCurrentItem(it)

class EditorActions(EditorAction):

    name = 'Editor Actions'
    #_is_builtin = True

    def run(self, chain, settings):
        
        ac = get_qaction_from_unique_name(chain.gui, settings['unique_name'])
        if ac:
            if DEBUG:
                prints('Editor Chains: Editor Actions: Found action: {}'.format(ac.text().replace('&', ''))) 

            cursor = Qt.WaitCursor
            if settings.get('disable_busy_cursor'):
                cursor = Qt.ArrowCursor
            
            QApplication.setOverrideCursor(cursor)
            try:
                ac.trigger()
                   
            finally:
                QApplication.restoreOverrideCursor()
            
        if DEBUG:
            prints('Editor Chains: Editor Actions: Finishing run action') 

    def validate(self, settings):
        gui = self.plugin_action.gui
        if not settings:
            return (_('Settings Error'), _('You must configure this action before running it'))
        if settings['unique_name'] not in allowed_unique_names(gui):
            return (_('Settings Error'), _('Action cannot be found: {}'.format(settings['unique_name'])))
        return True

    def config_widget(self):
        return ConfigWidget

Notes:

• If any of the plugins/actions you call require user interaction, you'll have to do it. There is no way to automate this.
• If you are familiar with Action Chains, this action is similar to an action called "Calibre Actions" in that plugin. I did not include this action in Editor Chains by default, nor I have plans to so. This is because in retrospect, including "Calibre Actions" in Actions Chains plugin is the main thing I regret, for several reasons.

Originally Posted by thiago.eec

1) Add the 'Check book' action

The action provided above should enable you to call check books, but it will not allow for automatic repair, unless you click the hyperlink to do so.

A more sophisticated version with options and automatic repair without clicking should be possible, I guess. But I am not familiar with that part of calibre code. If you are interested, you can add it as a custom action, through modules as in the action provided above.

Hope that helps.

Edit: Similar to Action Chains, this plugin provides an API for other plugins to provide actions for Editor Chains. A plugin developer needs the following to add actions:

1. Create a file called editor_chains.py in the root of the plugin folder.
2. Include all the actions he want in the above file.
3. Now in Editor Chains, the actions will appear under "Imported From Other Plugins" node.

The file editor_chains/actions/base.py provides instructions on how to write actions. You can also examine any of the actions in the editor_chain/actions for examples. The main method you need to implement is run(). If the you want the action to provide options for the user, also take a look at config_widget().

[thiago.eec]

Thank you very much. This custom action gave me full control of the editor!

Now, one last question:

I've always wanted to automate merging splitted files (filename_split0001.xhtml, filename_split0002.xhtml, etc). With the custom action you provided, I can access the 'Merge files' action, but I didn't find a way to select the files based on a regular expression, just like when using 'Search and replace' and the filename filter.

Is it possible?

[capink]

Originally Posted by thiago.eec

Thank you very much. This custom action gave me full control of the editor!

Now, one last question:

I've always wanted to automate merging splitted files (filename_split0001.xhtml, filename_split0002.xhtml, etc). With the custom action you provided, I can access the 'Merge files' action, but I didn't find a way to select the files based on a regular expression, just like when using 'Search and replace' and the filename filter.

Is it possible?

Not without an action to convert scopes to selection. That one if fairly straight forward:

from qt.core import (QWidget, QVBoxLayout, QGroupBox)

from calibre_plugins.editor_chains.actions.base import EditorAction
from calibre_plugins.editor_chains.scope import scope_names, ScopeWidget, validate_scope, scope_is_headless

class ConfigWidget(QWidget):
    def __init__(self, plugin_action):
        QWidget.__init__(self)
        self.plugin_action = plugin_action
        self._init_controls()

    def _init_controls(self):

        l = self.l = QVBoxLayout()
        self.setLayout(l)

        scope_groupbox = QGroupBox(_('Files to select'))
        l.addWidget(scope_groupbox)
        scope_l = QVBoxLayout()
        scope_groupbox.setLayout(scope_l)
        self.where_box = ScopeWidget(self, self.plugin_action, orientation='vertical',
                                     headless=self.plugin_action.gui is None,
                                     hide=['selected-text','open','current'])
        scope_l.addWidget(self.where_box)

        l.addStretch(1)
        self.setMinimumSize(400,200)

    def load_settings(self, settings):
        if settings:
            self.where_box.where = settings['where']

    def save_settings(self):
        settings = {}
        settings['where'] = self.where_box.where
        return settings

class SelectFiles(EditorAction):

    name = 'Select Files'
    headless = True

    def run(self, chain, settings, *args, **kwargs):
        names = scope_names(chain, settings['where'])
        chain.gui.file_list.select_names(names)

    def validate(self, settings):
        scope_ok = validate_scope(settings['where'])
        if scope_ok is not True:
            return scope_ok
        return True

    def config_widget(self):
        return ConfigWidget

    def is_headless(self, settings):
        return scope_is_headless(settings['where'])

The action above will select files based on filters. Note however, that merge in an interactive step which will require you to select the master file into which all other files will be merged.

Calibre uses a function called merge() to merge files. You can build an action around this function:

from qt.core import (QWidget, QVBoxLayout, QGroupBox)

from calibre.ebooks.oeb.base import OEB_DOCS, OEB_STYLES
from calibre.ebooks.oeb.polish.split import merge
from calibre.gui2.tweak_book import editors

from calibre_plugins.editor_chains.actions.base import EditorAction
from calibre_plugins.editor_chains.scope import scope_names, ScopeWidget, validate_scope, scope_is_headless


def spine_index(container, name):
    idx = 0
    for spine_name, is_linear in container.spine_names:
        idx +=1
        if name == spine_name:
            return idx

class ConfigWidget(QWidget):
    def __init__(self, plugin_action):
        QWidget.__init__(self)
        self.plugin_action = plugin_action
        self._init_controls()

    def _init_controls(self):

        l = self.l = QVBoxLayout()
        self.setLayout(l)

        scope_groupbox = QGroupBox(_('Files to merge'))
        l.addWidget(scope_groupbox)
        scope_l = QVBoxLayout()
        scope_groupbox.setLayout(scope_l)
        self.where_box = ScopeWidget(self, self.plugin_action, orientation='vertical',
                                     headless=self.plugin_action.gui is None,
                                     hide=['selected-text','open','current'])
        scope_l.addWidget(self.where_box)

        l.addStretch(1)
        self.setMinimumSize(400,200)

    def load_settings(self, settings):
        if settings:
            self.where_box.where = settings['where']

    def save_settings(self):
        settings = {}
        settings['where'] = self.where_box.where
        return settings

class MergeFiles(EditorAction):

    name = 'Merge Files'
    headless = True

    def run(self, chain, settings, *args, **kwargs):
        names = scope_names(chain, settings['where'])
        print(f'DEBUG: Merging Files\n       names: {names}')
        if len(names) < 2:
            print('Error: You need to select at least 2 files to merge')
            return
        container = chain.current_container
        spine_names = [name for name, is_linear in container.spine_names]
        if names[0] in set(container.manifest_items_of_type(OEB_DOCS)):
            category = 'text'
        elif names[0] in set(container.manifest_items_of_type(OEB_STYLES)):
            category = 'styles'
        else:
            print('Error: Can only merge files of categories: text or styles')
            return
        if category == 'text':
            if not set(names).issubset(set(container.manifest_items_of_type(OEB_DOCS))):
                print('Error: some files are your trying to merge are not of category text')
                return
            if not set(names).issubset(set(spine_names)):
                print('Error: Some files you are trying to merge not in spine')
                return
            names = sorted(names, key=lambda name: spine_index(container, name))
        else:
            if not set(names).issubset(set(container.manifest_items_of_type(OEB_STYLES))):
                print('Error: some files are your trying to merge are not of category styles')
                return
            names = sorted(names)
        master = names[0]
        print(f'DEBUG: Merging Files\n       master: {master}')
        merge(container, category, names, master)
        if master in editors:
            chain.boss.show_editor(master)

    def validate(self, settings):
        scope_ok = validate_scope(settings['where'])
        if scope_ok is not True:
            return scope_ok
        return True

    def config_widget(self):
        return ConfigWidget

    def is_headless(self, settings):
        return scope_is_headless(settings['where'])

Notes:

• The action above will merge text files according to their spine index. The master file into which others will be merged is the one with the lowest spine index. Style files are sorted alphabetically.
• This action is provided without any guarantee that it will work properly. I have not tested it much. You must test vigorously to ensure it merges files in correct order before accepting the results it provides