[Plugin] FolderIn and FolderOut - Folder input and output plugins for Sigil

[KevinH]

Updated: October 30, 2019
Current Version: "0.4.0"

See the attached: FolderIn_v030.zip and FolderOut_v040.zip

License/Copying: GNU LGPL Version 2 or Version 3 your choice. Any other license terms are only available directly from the author in writing.

The purpose of these plugins is to provide the ability for Sigil to:

FolderOut - copy files from an epub to a folder without any zipping

FolderIn - load epub files from a folder

These plugins were designed to allow ebook developers to more easily interface to git or some other version control system.

Note:
Both of these plugins should work properly with the upcoming Sigil 1.0 and later.

Note:
These plugins will not work with Python 2.7. They are designed to work with more recent versions of Sigil that have an embedded Python 3.X interpreter .

Note:
Since unzipped files in open folders can be changed by other tools, any embedded fonts are stored unobfuscated and both plugins will refuse to work with folders that have an encryption.xml file present in the META-INF folder to prevent obfuscation/unobfuscation mismatches from destroying the font files. Before producing the final epub using Sigil (or any other software), the proper embedded font subsetting and/or obfuscating should be performed depending on the license of the fonts embedded.

Changes
- added Sigil QuickLaunch Toolbar plugin icons for both FolderIn and FolderOut

[DiapDealer]

What are your thoughts on adding a property to be passed to plugins (via the sigil.cfg file) as to whether or not an epub is in an unsaved state or not? I'm thinking especially of input/output plugins. In the case of your FolderOut plugin, I think it could potentially cause problems/confusion if an OEBPS copy of an unsaved epub is exported.

At the very least, I think a warning should be added to the FolderOut plugin that unsaved changes can/will be exported. While we decide anyway.

I believe all input plugins already have a generic warning that the current epub is going to be overwritten--giving the user a chance to abort. I think it might be a good idea for output (or input) plugins to determine for themselves whether or not an unsaved epub warning is warranted. It would give users the ability to abort the output and save their changes first.

[KevinH]

I am not sure what the issue is. FolderIn is an input plugin and so you get the warning on Windows and Linux (Macs just open a new window in the current instance.). FolderOut or any output plugin has no impact on the saved/written epub, it is only given the state of the files inside Sigil. As FolderOut does not create a .epub file, only a folder, it can never lose or overwrite what was saved as .epub.

In PluginRunner **before** any plugin is started all state including any changes are written to files in Sigil's temp folder. In other words all tabs are flushed. This also happens before splitting and merging files, making reports, creating navs, etc. So all the plugin sees is the current state of the epub book inside Sigil. It never sees or knows about what is actually written to (saved) to the epub file.

All the dirty flag inside Sigil does is use the fact that something has changed rom the last save to alert them to do a save if they want to a .epub file.

So I guess I don't understand the issue. All plugins and all of Sigil itself only work on what is inside Sigil's temp folders. A Save-as or Save is needed to actually write these to a .epub file and the user determines if that happens or not.

[DiapDealer]

I understand all that. I'm just recognizing that some users may not REALIZE (or remember) that they've made edits to their epub before exporting via FolderOut, and then getting confused as to why their exported OEBPS doesn't match the contents of their epub (if they happen to discard their epub's edits when next saving).

Plus, when it comes to output plugins in general, I guess I'm foreseeing the potential for a plugin dev to not want to operate on unsaved edits. So rather than pop warnings that may not be necessary, I thought it might be handy for the plugin dev to be able to determine for themselves if their plugin was called on an epub that has unsaved edits.

As for FolderIn (or Input plugins in general), I just thought it might be possible to avoid the popup warning altogether if the current epub is already in a saved state, that's all.

If nobody else sees a need though, that's fine too.

[KevinH]

I am still not sure if I understand the issue. I view all input plugins as equivalent to the user opening an .epub file in Sigil, and all output plugins as equivalent to the user doing a Save-As inside Sigil.

Doing a Save-as is basically telling Sigil to take the current state inside Sigil (not what is in the .epub file itself) and save it. That is how I view output plugins.

Then the question becomes, should the successful completion of an output plugin clear the "dirty" flag. My thoughts were no since that flag just is there to protect the .epub file from lost changes to it.

As for running output plugins when the user has unsaved changes, I am happy it does, as it is equivalent to a Save-As. I would not want the user to have to do a Save or Save-as inside Sigil just to run an output plugin that should effectively be doing the same thing.

What am I missing? All parts of Sigil only work on the loaded epub stored inside temp, and never in the actual epub file until told to.

[KevinH]

It might help if you could provide an example of an output plugin that would need to know you have already done a save or save-as to a file before launching it?

Perhaps a flag that indicates if the current version inside of Sigil passes FlightCrew, or epubcheck or even just an enhanced sanity check might be useful for some conversion style output plugins? Take epub3-itizer as an example, it handles that by simply telling the user in the docs, to make sure the epub2 has passed flightcrew/epubcheck and they have renamed files to .xhtml before launch.

[DiapDealer]

Originally Posted by KevinH

I am still not sure if I understand the issue. I view all input plugins as equivalent to the user opening an .epub file in Sigil, and all output plugins as equivalent to the user doing a Save-As inside Sigil.

Doing a Save-as is basically telling Sigil to take the current state inside Sigil (not what is in the .epub file itself) and save it. That is how I view output plugins.

Not quite. A Save-As saves the current state to a new epub AND updates Sigil's current content with that of the new saved epub. The new epub is saved and the epub open in sigil is in a saved state that matches. Nothing is left in a dirty state with a Save As. An output plugin currently saves a new document (or folder in this case), but maintains it's current content connected to its previous save location. If the current content was unsaved before, it is still unsaved after. But that's neither here nor there RE my suggestion.

Originally Posted by KevinH

Then the question becomes, should the successful completion of an output plugin clear the "dirty" flag. My thoughts were no since that flag just is there to protect the .epub file from lost changes to it.

My thoughts are no, too. My suggestion has nothing to do with altering the current contents of the epub Sigil has open, or the saving of, or the clearing of Sigil's dirty flag. It has everything to with allowing a plugin to know that Sigil's dirty flag IS set.

Originally Posted by KevinH

As for running output plugins when the user has unsaved changes, I am happy it does, as it is equivalent to a Save-As. I would not want the user to have to do a Save or Save-as inside Sigil just to run an output plugin that should effectively be doing the same thing.

I'm happy it does that, too (though I've already pointed out the slight discrepancies to the "Save As" analogy). I'm talking about merely warning the user that their epub has unsaved edits before running the output plugin; not stopping them from running the plugin.

If you don't think your output plugin needs such a warning, then that's your choice. I merely thought it might be handy in case people hadn't noticed their epub had unsaved edits before they exported via FolderOut. But regardless of all that, I still find it reasonable that other plugin devs might want to know Sigil's dirty flag is set before processing anything.

[DiapDealer]

Originally Posted by KevinH

It might help if you could provide an example of an output plugin that would need to know you have already done a save or save-as to a file before launching it?

1) a user fatfingers, or inadvertantly makes an edit to a file in their epub
2) they use FolderOut to write the current contents (including the mistakes) to a folder
3) they play around with their epub some more, but decide to roll back to the last FolderOut version
4) they use FolderIn to overwrite the contents of the current epub with the version that contains mistakes they're unaware of.

Yes, using the standard diff tools religiously might save them from trouble, but not everyone using FolderOut will be using a VCS. It just seems relevant to me that a user KNOW that there's "uncommitted" changes that are going to be output beforehand.

If no one else agrees that it's necessary, then that's fine. I just want to make sure my point is being understood. It seems there's a significant disconnect in that regard.

[KevinH]

Okay, if we include the state of the dirty flag in the sigil.cfg file and add an interface to access it, should we also add the full path and/or filename of the current .epub file if known since Doitsu said he mght want it, and we could make those changes all together?

Are there any other pieces of info we should add?

[KevinH]

If we wanted, we could make that warning only in C++ PluginRunner code before launching any output plugins and not use sigil.cfg at all nor change any of the launcher/wrapper python code.

Alternatively, we pass the dirty flag to the plugin and let the plugin code produce the warning and handle it.

Are there any other uses for the dirty flag that would make it useful inside the plugin?

[DiapDealer]

Originally Posted by KevinH

Okay, if we include the state of the dirty flag in the sigil.cfg file and add an interface to access it, should we also add the full path and/or filename of the current .epub file if known since Doitsu said he mght want it, and we could make those changes all together?

That's fine by me.

Are there any other pieces of info we should add?

Seems to me there was something else I was considering, but I can't think of it at the moment.

[DiapDealer]

Originally Posted by KevinH

If we wanted, we could make that warning only in C++ PluginRunner code before launching any output plugins and not use sigil.cfg at all nor change any of the launcher/wrapper python code.

I don't really want to take the choice away from the plugin dev by making the warning mandatory whenever running output plugins with Sigil's dirty flag set. It's really only co-dependent input/output pairs of plugins where I envision there being a need for such a warning.

Alternatively, we pass the dirty flag to the plugin and let the plugin code produce the warning and handle it.

This is what I'm leaning toward. For the reasons mentioned above. I realize this
needs to be balanced against altering the plugin framework (which means plugins that utilize it might need need to juggle things in order to support earlier versions of Sigil that don't have the flag).

[jcsalomon]

Originally Posted by DiapDealer

Not quite. A Save-As saves the current state to a new epub AND updates Sigil's current content with that of the new saved epub. The new epub is saved and the epub open in sigil is in a saved state that matches. Nothing is left in a dirty state with a Save As. An output plugin currently saves a new document (or folder in this case), but maintains it's current content connected to its previous save location.

Something like the current “Save a Copy” option.

Perhaps input/output plugins should appear on the File menu:
New
Open…
Import › (submenu that’s currently under Plugins/Input)
Save
Save As…
Save A Copy…
Export › (submenu that’s currently under Plugins/Output)
Print Preview…
etc.…

This would make it clearer how these operations correspond to Sigil’s built-in functions.

[KevinH]

Sigil needs to fully function without plugins in the same way it did before we added the plugin capability. So all menu activation of plugin finctionality will be done through the plugin menu as it stands today.

That said, we may allow more toolbar buttons and plugin based icons to make access easier.

[KevinH]

Created versions 0.3.0 for both FolderIn and FolderOut to add in plugin.png icons