Adding a limited Automate Feature To Sigil

[KevinH]

All Good Points.

Perhaps, as you suggest instead of relying only on ShowMessageOnStatus bar to report things back, we do the following:

- when automate is first called we open a non-modal dialog with a log window
and a cancel button

- show the progress and results of the automate list step by step.
Sort of like the log window we generate for plugins.

But that will really clutter up the screen with more windows such as a non-modal dialog window plus lots of plugin related dialogs and output.

A better and simpler approach might be to modify the ShowMessageOnStatusBar routine to to append things to an automate log file as long as m_UsingAutomate is true.

Then once the automate list stops, a read only dialog of all the ShowStatusBarMessages is displayed with a OK button to continue. Sort of like the RepoManager logs.

This will all take some thinking and depends on developing a way to know if any non-modal dialog is closed or hidden so that automate operations can effectively block until those dialogs are complete.


Thanks!

KevinH

Originally Posted by BeckyEbook

Ad 1. English names will be the best. translations sometimes change, not complete, etc., so it's asking for trouble. The Automate option is intended for advanced users, so creating "automateXX.txt" script with English names should be OK.

Ad 2. If it is well described in the documentation – a good idea.

Ad 3. Of course, we wait until these dialogs are closed.

Ad 4. Hmmm ... A difficult matter, because in Saved Searches we can do various actions: Find, Replace, Replace/Find, Replace All.

Maybe something as:
* "F&R-Find:GroupNameFromSavedSearches" (calling a search in the whole group, probably quote marks will be necessary)

* "F&R-ReplaceAll:GroupNameFromSavedSearches/Single Sample Replace" (Replace All In Single Search In Saved Searches)

-----

I would add an optional AutomateXX operation report (message box? logfile?):

Function Name or Plugin Name 1: passed
Function Name or Plugin Name 2: passed
Function Name or Plugin Name 3: passed with errors
Aborted by user

Personally, I built a new version and played around with this option (only plugins so far).
Despite my earlier doubts, it might come in handy for some repetitive operations on each file.

That's why I noticed that I missed a summary/report in which I could see that all the plugins "did their job" without errors. Now I have to look at the statusbar, because it is probably only there that we can see what is going on.

[BeckyEbook]

I still have to think about the "Legacy" version.

Here Material & Fluent.

[KevinH]

Originally Posted by BeckyEbook

I still have to think about the "Legacy" version.

Here Material & Fluent.

Thank You!

I will integrate them tomorrow into master.

KevinH

[KevinH]

Okay support has been pushed for the following to master:

- BeckyEbook's fluent and material icons
- Support for a Automate Log
- Fixed Validation Results to properly report when No Problems Found
- Added support for the following C++ MainWindow tools:

• UpdateManifestProperties
• Save
• WellFormedCheckEpub
• MendPrettifyHTML
• MendHTML
• ValidateStylesheetsWithW3C
• RepoCommit (Checkpoint)
• GenerateNCXGuideFromNav
• RemoveNCXGuideFromEpub3
• DeleteUnusedMedia
• DeleteUnusedStyles
• StandardizeEpub (Restructure Epub to Sigil Norm)
• SplitOnSGFSectionMarkers
• AddCover

Still no support for any non-modal dialog based tools such as SpellCheck, Saved Searches, Reports, EditTOC, EditMetadata but even without them this proof of concept should hopefully prove useful enough for serious users/devs to make use of.

For example the equivalent of a Publish button for Epub3 should now be possible:

Contents of automate01.txt

# start with Checkpoint save to prevent any data loss
RepoCommit

# run a basic well-formed test before running any advanced tools
WellFormedCheckEpub

# prettify any code and fix any remaining html errors
MendPrettifyHTML

# clean up
DeleteUnusedMedia
DeleteUnusedStyles

# make sure styles are valid
ValidateStylesheetsWithW3C

# make en epub3 that works with epub2 only readers
GenerateNCXGuideFromNav
#RemoveNCXGuideFromEpub3

UpdateManifestProperties
EpubCheck
Save

[KevinH]

If you build you own on Linux, Mac, or Windows, please give master a try and let me know your thoughts.

If it looks useable, we can use deploy to create both Windows and macOS builds that we can make available to serious users and testers for more feedback.

If this turns out to not really be useful, we can simply strip it out easily enough.

[RbnJrg]

Originally Posted by KevinH

For example the equivalent of a Publish button for Epub3 should now be possible:

Contents of automate01.txt

# start with Checkpoint save to prevent any data loss
RepoCommit

# run a basic well-formed test before running any advanced tools
WellFormedCheckEpub

# prettify any code and fix any remaining html errors
MendPrettifyHTML

# clean up
DeleteUnusedMedia
DeleteUnusedStyles

# make sure styles are valid
ValidateStylesheetsWithW3C

# make en epub3 that works with epub2 only readers
GenerateNCXGuideFromNav
#RemoveNCXGuideFromEpub3

UpdateManifestProperties
EpubCheck
Save

That routine is great!! Sigil 1.8 will be incredible!

[Doitsu]

Originally Posted by KevinH

If you build your own on Linux, Mac, or Windows, please give master a try and let me know your thoughts.

I just tested this on my Linux machine, and it appears to be working fine.

I only have one minor nitpick: the automation script will fail, if the automation list contains actions that can't be selected for epub2 books.

For example:

GenerateNCXGuideFromNav
RemoveNCXGuideFromEpub3
UpdateManifestProperties

Ideally, the automation tool should check the version number and only execute them, if the current book is an epub3 book.

[KevinH]

Thanks for testing!

Those tools are for epub3 only and should fail on epub2 in my opinion.
Just like there are plugins like flightcrew (and other plugins like ePub3-itizer) that work only on epub2 and should fail on epub3.

I don't even have a way for epub version specific plugins to identify themselves so checking epub version before running a command/plugin would not be straightforward.

Couldn't users just create separate automate lists for epub2 and epub3?

The limit of 3 automate list could easily be raised to 10 if that would help.

[Thasaidon]

Originally Posted by KevinH

Thanks for testing!


Couldn't users just create separate automate lists for epub2 and epub3?

Sounds workable and fair to me.

Originally Posted by KevinH

The limit of 3 automate list could easily be raised to 10 if that would help.

Three lists would probably be enough for me but if people regularly work with both wpub2 and epub3 it might be useful to have a few more lists.

[Doitsu]

Originally Posted by KevinH

1. How should we deal with intl Sigil user/devs in these automate lists if at all?
In other words should we be treating them like shell/bat commands which are all English based like most programming and scripting languages?

I also think that the commands shouldn't be localized.

Originally Posted by KevinH

3. How to deal with non-modal dialogs like Search Editor / SpellCheck, Edit TOC, Reports, Edit Metadata, etc? Should we wait until these dialogs are closed by the user before proceeding to the next command in the automate list?

IMHO, it doesn't make sense to automate those tasks anyway.

Originally Posted by KevinH

The limit of 3 automate lists could easily be raised to 10 if that would help.

IMHO, 3 automate lists should be more than enough even for professional ebook creators.

[KevinH]

Originally Posted by Doitsu

Originally Posted by KevinH View Post
3. How to deal with non-modal dialogs like Search Editor / SpellCheck, Edit TOC, Reports, Edit Metadata, etc? Should we wait until these dialogs are closed by the user before proceeding to the next command in the automate list?


IMHO, it doesn't make sense to automate those tasks anyway.

I was wondering about that.

The only advantage I could see to supporting dialog based tools is to have an automate list end with one of those but not wait for completion as using any of them requires significant user interaction.

I did some looking and EditMetadata, Generate Table of Contents, and Edit Table of Contents appear to be fully modal so progress will block until they are closed anyway.

And Create HTML Table of Contents, Spellcheck, Saved Searches (F&R) and Reports are fully non-modal (progress will not block unless we wait for them to close) and you can leave them to edit inside the ebook at will using other Sigil features.

Furthermore, closing them is not needed to signify you are done with them as they may remain open until you use them next.

So it is a big question as to what value automating these dialog based tools would be anyway outside of ending in one of them to save a single keystroke shortcut, icon press, or menu click.

Given how flexible plugins are a lot of more complex conditionalized changes can be made including very advanced replacements in a plugin if needed.

So how in general do we deal with these dialog tools that require lots of input from the user itself to happen?

Do we try to automate them and if so to what benefit?

Are they even needed given that plugins and saved searches on groups can be done and run easily with just a few clicks/keystrokes/icons.


Open to any thoughts as to if/why they should be supported and how they will benefit people specifically.

I am starting to agree with Doitsu does here but would like to hear from others as well.

[Binchen]

Enhance the plugin category from edit/validate/output? Or only plugins which are for validating can be automated?

Just a quick idea...

[KevinH]

Plugin validation can be epub version specific as well. All plugins are already supported. The question was about the Sigil built-in C++ tools that are Dialog based tools.

[phillipgessert]

I earlier wondered if it would be feasible (or even possible) to optionally set some sort of “auto-confirm” on some of these that wait for input. I stopped myself suggesting it because I realize that’s kind of a risky option to unleash. Personally I’d find it handy.

And I agree that ten automations does sound like a lot, though I could see some benefit wrt folks that might like their automations a little more bite-sized.

[KevinH]

The following automation changes have been made:

- Epub3 only C++ tools will now succeed but with a log message that they were skipped when run on epub2. Hopefully this will make just 3 automation lists more feasible.

- I have extended the list of known C++ MainWindow Command Line tools with "GenerateTOC", and "CreateHTMLTOC" and have made the "GenerateTOC" automate routine skip the Heading Selector dialog.

The full list of C++ MainWindow Tools supported in master is now:

• UpdateManifestProperties
• Save
• WellFormedCheckEpub
• MendPrettifyHTML
• MendHTML
• ValidateStylesheetsWithW3C
• RepoCommit (Checkpoint)
• GenerateNCXGuideFromNav
• RemoveNCXGuideFromEpub3
• DeleteUnusedMedia
• DeleteUnusedStyles
• StandardizeEpub (Restructure Epub to Sigil Norm)
• SplitOnSGFSectionMarkers
• AddCover
• GenerateTOC (will create from existing headers without the Heading Editor functionality)
• CreateHTMLTOC

I have also modified the PluginRunner but this impacts macOS users of input plugins only.

On macOS input plugins always opened a new Window which would break automation.

Now macOS will replace the current book with the one from the input plugin in the same window just as Linux and Windows has always done. This is now similar to how we have implemented Restore from Checkpoint on macOS.

All of this was just pushed to master.