Adding a limited Automate Feature To Sigil - Page 4

KevinH · 08-19-2021, 02:10 PM

Hi All,

If anyone is interested in testing this new automate feature seeing if it is useful and providing feedback, bug reports, etc; And if you use Windows (64 bit) or macOS:

Please pm "KevinH" here on MobileRead and I will pm you back with a link to a pre-release build of Sigil master for you to download and install.

Bandwidth on these builds is limited so only reply if you are serious about helping us out with this new feature.

Thanks!

KevinH

KevinH · 08-20-2021, 03:27 PM

Hi,

I have only had one volunteer tester for the Automate feature so far so if you can help, please pm me.

Thanks,

KevinH

Quote:

Originally Posted by KevinH

Hi All,

If anyone is interested in testing this new automate feature seeing if it is useful and providing feedback, bug reports, etc; And if you use Windows (64 bit) or macOS:

Please pm "KevinH" here on MobileRead and I will pm you back with a link to a pre-release build of Sigil master for you to download and install.

Bandwidth on these builds is limited so only reply if you are serious about helping us out with this new feature.

Thanks!

KevinH

KevinH · 08-24-2021, 11:21 AM

More Automate Related Features added:

Based on comments and requests from the testers of this proposed automate feature, the following improvements and changes have been made:

- Added Support for reformatting all CSS to Multiple Line format

- Added Support for reformatting all CSS to Single Line format

- Added Support for running a named Saved Search as ReplaceAll

- Changed the Automate Toolbar to use 3 QToolButtons that when clicked and held will pop-up a menu to allow editing Automate Lists

- Added a new AutomateEditor and support dialogs to allow creation and editing of the 3 automate lists that is based on the approach used in the MetaData editor with buttons
to choose a supported Tool command or an existing/installed plugin, and arrow keys to easily move lines.

All of this has been pushed to master.

To help document things:

The 3 Automate Lists are stored inside the Sigil preferences folder:
automate01.txt
automate02.txt
automate03.txt

Each file is utf-8 encoded with one tool command or plugin name per line.
A leading # indicates a comment.

To run an automate list, simply click on the corresponding robotic arm icon in the Automate Toolbar.

To edit/create an automate list, simply click and hold on the corresponding robotic arm icon in the Automate Toolbar and select Edit from the resulting pop-up menu.

All currently installed Plugins are supported by name.

The Tools now supported:

AddCover
CreateHTMLTOC
DeleteUnusedMedia
DeleteUnusedStyles
GenerateNCXGuideFromNav
GenerateTOC (will create from existing headers without the Heading Editor)
MendPrettifyHTML
MendHTML
ReformatCSSMultipleLines
ReformatCSSSingleLines
RemoveNCXGuideFromEpub3
RepoCommit (Checkpoint)
RunSavedSearchReplaceAll [full name of a saved search]
Save
SplitOnSGFSectionMarkers
StandardizeEpub (Restructure Epub to Sigil Norm)
UpdateManifestProperties
ValidateStylesheetsWithW3C
WellFormedCheckEpub

I will be sending new test build links out to those who volunteered to test for further feedback and bug reports.

If interested to help out and willing to test and provide feedback, please PM "KevinH" and I will send you a link.

Thanks to all who have provided input on this process. Hopefully it will result in something useful for our more advanced users.

People who build from source your input is, as always, welcomed. Most importantly I am looking for feedback on the following:

Is this automate feature truly useful as implemented?

What additional Tool truly *needs* to be added (if any)?

Is the built-in Automate List Editor adequate given users can
use any text editor to accomplish the same thing easily enough?

BeckyEbook · 08-24-2021, 12:16 PM

Ad RunSavedSearchReplaceAll: How to run Saved Search from inside the group?

EDIT: I already know. RunSavedSearchReplaceAll GroupName/SavedSearchName

Quote:

Is this automate feature truly useful as implemented?

Yes.

Quote:

What additional Tool truly *needs* to be added (if any)?

Probably not, but I will go over all the functions carefully and maybe I will find something.

Quote:

Is the built-in Automate List Editor adequate given users can
use any text editor to accomplish the same thing easily enough?

The new editor for automateXX.txt files was a surprise for me and I like it very much, because I don't have to remember the exact names, but choose them calmly from the list and arrange them in the right order.

KevinH · 08-24-2021, 12:29 PM

The currently supported RunSavedSearchReplaceAll accepts one parameter on the line which represents the full name of the saved search or saved search group. Remember Group names end with a "/" just like you see as the tooltip when hovering over a group or search in the SavedSearch Dialog.

For example I have a search group called AutomateTest with two searches in the group named test1 and test2.

If I add the command:

RunSavedSearchReplaceAll AutomateTest/

it will run the group (both tests)

instead if I do:

RunSavedSearchReplaceAll AutomateTest/test2

it will run just the test2

DNSB · 08-24-2021, 12:35 PM

Quote:

Originally Posted by KevinH

Is this automate feature truly useful as implemented?

I am finding it very useful.

Quote:

Originally Posted by KevinH

Is the built-in Automate List Editor adequate given users can use any text editor to accomplish the same thing easily enough?

It was a very pleasant surprise. Compared to remembering the function/plugin names and mistyping them, it makes creating and modifying actions much easier. Automating the automation tool.

DiapDealer · 08-24-2021, 12:36 PM

Just checked out the latest. Fun stuff!

It's probably only cosmetic, but the automate log seems to contain spurious entries about being "skipped since not an epub3."

Code:

DeleteUnusedStyles running
DeleteUnusedStyles skipped since not an epub3
MendPrettifyHTML running
MendPrettifyHTML skipped since not an epub3
ReformatCSSMultipleLines running
This EPUB does not contain any CSS stylesheets to reformat.
ReformatCSSMultipleLines skipped since not an epub3
RemoveNCXGuideFromEpub3 running
RemoveNCXGuideFromEpub3 skipped since not an epub3
RepoCommit running
Checkpoint saved.
RepoCommit skipped since not an epub3
StandardizeEpub running
Restructure completed.
StandardizeEpub skipped since not an epub3
UpdateManifestProperties running
UpdateManifestProperties skipped since not an epub3
ValidateStylesheetsWithW3C running
This EPUB does not contain any CSS stylesheets to validate.
ValidateStylesheetsWithW3C skipped since not an epub3
WellFormedCheckEpub running
WellFormedCheckEpub skipped since not an epub3
Validation Tool Reported No Problems Found
EpubCheck running
Validation Tool Reported No Problems Found
Automation List Completed

With the exception of RemoveNCXGuideFromEpub3 and UpdateManifestProperties, none of my tasks should have been epub3 specific. Plus they all seemed to be successful (even if many had nothing to do). Checkpoints were created, MendPrettify was run, CSS was reformatted, etc...

KevinH · 08-24-2021, 12:39 PM

Yes must be a cosmetic bug. I am on it.

Yes, it was cosmetic only, and now fixed in master with a new deploy in case you want to play with it.

Thanks!

Kevin

Quote:

Originally Posted by DiapDealer

Just checked out the latest. Fun stuff!

It's probably only cosmetic, but the automate log seems to contain spurious entries about being "skipped since not an epub3."

Code:

DeleteUnusedStyles running
DeleteUnusedStyles skipped since not an epub3
MendPrettifyHTML running
MendPrettifyHTML skipped since not an epub3
ReformatCSSMultipleLines running
This EPUB does not contain any CSS stylesheets to reformat.
ReformatCSSMultipleLines skipped since not an epub3
RemoveNCXGuideFromEpub3 running
RemoveNCXGuideFromEpub3 skipped since not an epub3
RepoCommit running
Checkpoint saved.
RepoCommit skipped since not an epub3
StandardizeEpub running
Restructure completed.
StandardizeEpub skipped since not an epub3
UpdateManifestProperties running
UpdateManifestProperties skipped since not an epub3
ValidateStylesheetsWithW3C running
This EPUB does not contain any CSS stylesheets to validate.
ValidateStylesheetsWithW3C skipped since not an epub3
WellFormedCheckEpub running
WellFormedCheckEpub skipped since not an epub3
Validation Tool Reported No Problems Found
EpubCheck running
Validation Tool Reported No Problems Found
Automation List Completed

With the exception of RemoveNCXGuideFromEpub3, none of my tasks should have been epub3 specific. Plus they all seemed to be successful (even if many had nothing to do). Checkpoints were created, MendPrettify was run, CSS was reformatted, etc...

phillipgessert · 08-24-2021, 05:52 PM

Quote:

Originally Posted by BeckyEbook

The new editor for automateXX.txt files was a surprise for me and I like it very much, because I don't have to remember the exact names, but choose them calmly from the list and arrange them in the right order.

Yeah, I also really like this. It actually surfaces features I'd not have thought of using, necessarily--a pretty cool side effect.

RbnJrg · 08-24-2021, 06:17 PM

Not an error, but in order to see how Sigil can process an erroneous command, I opened automate01.txt and I included by hand the name of the plugin "XX_Everything" (of course, the plugin is imaginary, doesn't exist). Well, I received the following message:

Code:

StandardizeEpub running
Restructure completed.
StandardizeEpub skipped since not an epub3 <<
RemoveInLineStyles running
ReformatCSSMultipleLines running
ReformatCSSMultipleLines skipped since not an epub3 <<
XX_Everything running
Missing or unknown plugin or tool: XX_Everything
Automation List Failed << Sigil doesn't process the commands that follow

Maybe it could be preferable that Sigil -when there is an error- instead of aborting the list, it will be able to offers the users the option to continue or to abort.

KevinH · 08-24-2021, 10:03 PM

FWIW, I do think terminating the list is the right thing to do for non-existent plugins, plugins that report failures, and Tools that report failure. Continuing would be dangerous and the log file will show you exactly what went wrong so you can fix the Automate list, then revert to the last save Checkpoint or reload the file and run the corrected list.

Quote:

Originally Posted by RbnJrg

Not an error, but in order to see how Sigil can process an erroneous command, I opened automate01.txt and I included by hand the name of the plugin "XX_Everything" (of course, the plugin is imaginary, doesn't exist). Well, I received the following message:

Code:

StandardizeEpub running
Restructure completed.
StandardizeEpub skipped since not an epub3 <<
RemoveInLineStyles running
ReformatCSSMultipleLines running
ReformatCSSMultipleLines skipped since not an epub3 <<
XX_Everything running
Missing or unknown plugin or tool: XX_Everything
Automation List Failed << Sigil doesn't process the commands that follow

Maybe it could be preferable that Sigil -when there is an error- instead of aborting the list, it will be able to offers the users the option to continue or to abort.

Capricorn · 08-25-2021, 01:54 AM

This sounds wonderful. I have no idea how easy it is to implement, as I am not a programmer.
If possible, it would be like an early Sigil christmas present.
THANK YOU.

BeckyEbook · 08-25-2021, 02:47 AM

Maybe add additional checking the contents of the automateXX.txt file (eg in the "Automate List Editor" window or as a separate option in the submenu)?
Fast parsing and checking if the names of tools, plugins, and Saved Search are correct.

KevinH · 08-25-2021, 06:35 AM

Quote:

Originally Posted by BeckyEbook

Maybe add additional checking the contents of the automateXX.txt file (eg in the "Automate List Editor" window or as a separate option in the submenu)?
Fast parsing and checking if the names of tools, plugins, and Saved Search are correct.

I am thinking these Automate lists are more like simple .bat files. If you use the editor to create them you will aways have valid commands, but that still does 't mean they will run successfully. So a validator is a bit of overkill in my opinion when you just revert the checkpoint and run again if needed.

Turtle91 · 08-25-2021, 10:15 AM

I noticed the "RemoveNCXGuideFromEpub3" function and was wondering if it is also possible to change an ePub3 to an ePub2.

Currently I manually open the opf from the zip file and change the version=3 to version=2 then save and reopen. Not a difficult thing to do, just time consuming.

I'm thinking this functionality would require closing the epub but leave the sigil shell open to run the script, then reopening the epub...

08-19-2021, 02:10 PM	#46
KevinH Sigil Developer Posts: 9,071 Karma: 6361556 Join Date: Nov 2009 Device: many	New Automate Feature Testers Wanted Hi All, If anyone is interested in testing this new automate feature seeing if it is useful and providing feedback, bug reports, etc; And if you use Windows (64 bit) or macOS: Please pm "KevinH" here on MobileRead and I will pm you back with a link to a pre-release build of Sigil master for you to download and install. Bandwidth on these builds is limited so only reply if you are serious about helping us out with this new feature. Thanks! KevinH

08-24-2021, 11:21 AM	#48
KevinH Sigil Developer Posts: 9,071 Karma: 6361556 Join Date: Nov 2009 Device: many	More Automate Related Features added: Based on comments and requests from the testers of this proposed automate feature, the following improvements and changes have been made: - Added Support for reformatting all CSS to Multiple Line format - Added Support for reformatting all CSS to Single Line format - Added Support for running a named Saved Search as ReplaceAll - Changed the Automate Toolbar to use 3 QToolButtons that when clicked and held will pop-up a menu to allow editing Automate Lists - Added a new AutomateEditor and support dialogs to allow creation and editing of the 3 automate lists that is based on the approach used in the MetaData editor with buttons to choose a supported Tool command or an existing/installed plugin, and arrow keys to easily move lines. All of this has been pushed to master. To help document things: The 3 Automate Lists are stored inside the Sigil preferences folder: automate01.txt automate02.txt automate03.txt Each file is utf-8 encoded with one tool command or plugin name per line. A leading # indicates a comment. To run an automate list, simply click on the corresponding robotic arm icon in the Automate Toolbar. To edit/create an automate list, simply click and hold on the corresponding robotic arm icon in the Automate Toolbar and select Edit from the resulting pop-up menu. All currently installed Plugins are supported by name. The Tools now supported: AddCover CreateHTMLTOC DeleteUnusedMedia DeleteUnusedStyles GenerateNCXGuideFromNav GenerateTOC (will create from existing headers without the Heading Editor) MendPrettifyHTML MendHTML ReformatCSSMultipleLines ReformatCSSSingleLines RemoveNCXGuideFromEpub3 RepoCommit (Checkpoint) RunSavedSearchReplaceAll [full name of a saved search] Save SplitOnSGFSectionMarkers StandardizeEpub (Restructure Epub to Sigil Norm) UpdateManifestProperties ValidateStylesheetsWithW3C WellFormedCheckEpub I will be sending new test build links out to those who volunteered to test for further feedback and bug reports. If interested to help out and willing to test and provide feedback, please PM "KevinH" and I will send you a link. Thanks to all who have provided input on this process. Hopefully it will result in something useful for our more advanced users. People who build from source your input is, as always, welcomed. Most importantly I am looking for feedback on the following: Is this automate feature truly useful as implemented? What additional Tool truly needs to be added (if any)? Is the built-in Automate List Editor adequate given users can use any text editor to accomplish the same thing easily enough? Last edited by KevinH; 08-24-2021 at 11:52 AM. Reason: update for additional testers and better documentation

08-24-2021, 12:29 PM	#50
KevinH Sigil Developer Posts: 9,071 Karma: 6361556 Join Date: Nov 2009 Device: many	The currently supported RunSavedSearchReplaceAll accepts one parameter on the line which represents the full name of the saved search or saved search group. Remember Group names end with a "/" just like you see as the tooltip when hovering over a group or search in the SavedSearch Dialog. For example I have a search group called AutomateTest with two searches in the group named test1 and test2. If I add the command: RunSavedSearchReplaceAll AutomateTest/ it will run the group (both tests) instead if I do: RunSavedSearchReplaceAll AutomateTest/test2 it will run just the test2 Last edited by KevinH; 08-24-2021 at 12:34 PM.

08-24-2021, 06:17 PM	#55
RbnJrg Wizard Posts: 1,876 Karma: 8821117 Join Date: Mar 2013 Location: Rosario - Santa Fe - Argentina Device: Kindle 4 NT	Not an error, but in order to see how Sigil can process an erroneous command, I opened automate01.txt and I included by hand the name of the plugin "XX_Everything" (of course, the plugin is imaginary, doesn't exist). Well, I received the following message: Code: StandardizeEpub running Restructure completed. StandardizeEpub skipped since not an epub3 << RemoveInLineStyles running ReformatCSSMultipleLines running ReformatCSSMultipleLines skipped since not an epub3 << XX_Everything running Missing or unknown plugin or tool: XX_Everything Automation List Failed << Sigil doesn't process the commands that follow Maybe it could be preferable that Sigil -when there is an error- instead of aborting the list, it will be able to offers the users the option to continue or to abort.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Feature requested for Sigil 1.7	RbnJrg	Sigil	23	08-06-2021 08:44 AM
Feature Requests for Adding Books	toomuchreading	Library Management	5	03-17-2015 03:15 PM
Parity feature with Sigil	roger64	Editor	8	03-17-2014 11:28 AM
New to "Read in Store" feature: Limited Selection	negris123	Barnes & Noble NOOK	4	06-09-2011 01:08 PM
New Inkmesh Feature: Limited Time Free Ebooks	anurag	News	11	02-25-2010 02:43 PM

08-25-2021, 01:54 AM	#57
Capricorn Belgian Pommes Frites Posts: 131 Karma: 35038 Join Date: Jan 2012 Device: Pocketbook Touch HD	This sounds wonderful. I have no idea how easy it is to implement, as I am not a programmer. If possible, it would be like an early Sigil christmas present. THANK YOU.

08-25-2021, 02:47 AM	#58
BeckyEbook Guru Posts: 899 Karma: 3501166 Join Date: Jan 2017 Location: Poland Device: Various	Maybe add additional checking the contents of the automateXX.txt file (eg in the "Automate List Editor" window or as a separate option in the submenu)? Fast parsing and checking if the names of tools, plugins, and Saved Search are correct.

08-25-2021, 10:15 AM	#60
Turtle91 A Hairy Wizard Posts: 3,395 Karma: 20212733 Join Date: Dec 2012 Location: Charleston, SC today Device: iPhone 15/11/X/6/iPad 1,2,Air & Air Pro/Surface Pro/Kindle PW & Fire	I noticed the "RemoveNCXGuideFromEpub3" function and was wondering if it is also possible to change an ePub3 to an ePub2. Currently I manually open the opf from the zip file and change the version=3 to version=2 then save and reopen. Not a difficult thing to do, just time consuming. I'm thinking this functionality would require closing the epub but leave the sigil shell open to run the script, then reopening the epub...