Possible To-Do List for Future Sigil Releases Post Sigil 1.8

[KevinH]

Thanks for testing. No, as I said, it was caused by unmanifested files inside META-INF existing along side empty folders.

[KevinH]

And BeckyEbook is right, Adding a file in Sigil, adds it to the epub manifest and so adding an ibooks xml file that way is incorrect. We should probably prevent it somehow but people could just add it under a different name and rename (still incorrect) and etc, The plugin is the correct way to add it.

[DiapDealer]

Originally Posted by KevinH

We should probably prevent it somehow but people could just add it under a different name and rename (still incorrect) and etc, The plugin is the correct way to add it.

I'm in favor of whatever solution doesn't require us to add even more code to support a single, purely-proprietary file that, as I understand it, is becoming increasingly unnecessary.

[democrite]

Hello,

I'm not sure how difficult this would be. In cases when the file structure hasn't changed, e.g. same file names and none added or deleted, or perhaps more cases, maybe some kind of optimized saving where only changes are saved? Large files with many images can take a while.

[KevinH]

That is already the practice for image files. They are not cached as they are binary not text resources that are only written to if Open With is used. They still need to be compressed and written to the epub being saved. Again, the Zip file is not a filesystem. It needs to be created when saving is done.

How many image files are we talking about here?

[democrite]

Any EPUB in the hundreds of MB. I am aware of the use of a temp folder. . I was just thinking perhaps - if it is a good idea, I don’t know - updating the EPUB in place with only changes, at times when possible. Or keeping a saved working copy that is updated as such and then replaces original.

Some of us, I’m not sure how many, are more comfortable with somewhat frequent saves.

Whether problematic in some cases, unsure. But would be nice someday if it can be done. If for example, at relevant times on disk EPUB file structure was stored and compared when saving.

[phillipgessert]

I had a thought today for a possible feature. Would it be possible/interesting/widely useful enough to allow a user to load in e.g. a TSV file with a big batch of find/replace entries to be executed at once? My thinking is that it could be useful in executing revisions. Something like:

<p>The quick brown box chases the lazy dog</p>    <p>The quick brown fox chases the lazy dog</p>
<p>Let the rat out of the bag</p>    <p>Let the cat out of the bag</p>

[KevinH]

Sigil's checkpointing feature does that. It saves the complete unpacked epub state in an underlying git repo. It is much faster than saving after the initial check in.

Originally Posted by democrite

Any EPUB in the hundreds of MB. I am aware of the use of a temp folder. . I was just thinking perhaps - if it is a good idea, I don’t know - updating the EPUB in place with only changes, at times when possible. Or keeping a saved working copy that is updated as such and then replaces original.

Some of us, I’m not sure how many, are more comfortable with somewhat frequent saves.

Whether problematic in some cases, unsure. But would be nice someday if it can be done. If for example, at relevant times on disk EPUB file structure was stored and compared when saving.

[democrite]

Originally Posted by phillipgessert

I had a thought today for a possible feature. Would it be possible/interesting/widely useful enough to allow a user to load in e.g. a TSV file with a big batch of find/replace entries to be executed at once? ...

Someday some expansion of automation lists, or some revised method of automation could be nice.

Various things I change with all purchased EPUBs. Such takes a bit of time. Some may not be easily automated as of now.

- replace cover w/same image, or set semantics
- remove justification
- if base text isn't 100%/1em, change and scale other styles
- remove font-family from body text styles
- remove line-height from non-headers
- convert to EPUB3 if not yet, change page # tags (e.g. if specified as anchor) to EPUB3, run pagelist plugin
- delete common fonts such as LinLibertine, Charis, etc.
- to make navigation easier for larger books, add to h2+ headers link back to chapter id in html toc
- if no html toc, create - some option to add to NAV/TCX if not there, like hold down alt/option w/menu selection?

possibly more I can't think of.

For some of that, unsure if asked before. Could be some amount of work. Possibly far off as maybe some future feature? As one can modify styles in some web inspector, some style panel where one could easily add or remove attributes to style at cursor point, oh would be so nice.

[phillipgessert]

Originally Posted by democrite

Someday some expansion of automation lists, or some revised method of automation could be nice.

Various things I change with all purchased EPUBs. Such takes a bit of time. Some may not be easily automated as of now.

- replace cover w/same image, or set semantics
- remove justification
- if base text isn't 100%/1em, change and scale other styles
- remove font-family from body text styles
- remove line-height from non-headers
- convert to EPUB3 if not yet, change page # tags (e.g. if specified as anchor) to EPUB3, run pagelist plugin
- delete common fonts such as LinLibertine, Charis, etc.
- to make navigation easier for larger books, add to h2+ headers link back to chapter id in html toc
- if no html toc, create - some option to add to NAV/TCX if not there, like hold down alt/option w/menu selection?

possibly more I can't think of.

For some of that, unsure if asked before. Could be some amount of work. Possibly far off as maybe some future feature? As one can modify styles in some web inspector, some style panel where one could easily add or remove attributes to style at cursor point, oh would be so nice.

Ok, but no offense, this doesn’t have much to do with what I said or suggested. I’m not giving you a hard time, just don’t want my idea to get buried. I’m not proposing a change to the automation features and I don’t think what I’m saying has much to do with the way automation features are implemented. More of an addition to find/replace. A way to easily do a bunch of stuff to one book, not the same thing over and over again to a bunch of books.

[democrite]

Thinking of search and replace in general, or other forms of automation, a greater thinking of how such might be done may lead to revisiting any aspect of how they are done now. Maybe some external file as you suggest, maybe instead labeled groups of saved searches (e.g. for me many replacements or use of TagMechanic differ by publisher), or something else.

For me, I tend to always try to think of what is easiest to create, maintain, and whatever takes the least amount of mouse clicks or keystrokes, e.g. like being able to go to css definition from cursor position, refactoring/removing some tag, attribute, class, etc. from cursor position also would be easiest. Though I understand such features compared to other IDEs or editors come from commercial or larger projects with a larger team, perhaps difficult for Sigil itself.

[Doitsu]

Originally Posted by phillipgessert

I had a thought today for a possible feature. Would it be possible/interesting/widely useful enough to allow a user to load in e.g. a TSV file with a big batch of find/replace entries to be executed at once?

You can import tab-delimited text files (and .csv files) in the Saved Searches dialog box.

The format is:

group_name/search_name\tfind\treplace

(Obviously, you'll need to replace \t with an actual tab.)

For example:

test/test1	colour	color
test/test2	centre	center

[phillipgessert]

Originally Posted by Doitsu

You can import tab-delimited text files (and .csv files) in the Saved Searches dialog box.

The format is:

group_name/search_name\tfind\treplace

(Obviously, you'll need to replace \t with an actual tab.)

For example:

test/test1	colour	color
test/test2	centre	center

Whelp, I might’ve guessed it was already possible! Thank you for the tip!

[KevinH]

You do know that all of that can be done via a custom Sigil plugin right now.
So instead of expanding and creating our our automation language, it is much better to simply let users handle it in a well-established scripting language like python via our Sigil plugin system.

And, in case I was not clear earlier when I said "a zip file is NOT a file system", a zip is a write once type of thing. Any modified files have to be written to the end (duplicated) and then the old record zeroed out. Most zip implementations just recreate the zip (smaller and faster) than trying to modify compressed file contents in place since any size growth means a complete rewrite anyway.

You really need to think about unzipping your files and working with them in git using FolderIn/FolderOut or your own custom input and output plugins. That way only modified files are added to the repo after the initial checkin, changes can be made readily apparent, tags that represent versions/revisions can be made, modification dates easily kept, etc.

This is why we created the Sigil Checkpoint feature using dulwich (pure python based simple/limited git implementation) so that state, fast saves, seeing modifications, etc can all easily be done.

You are thinking of (or used to?) unpacking a zip to a file system, working on individual files (no caching, no auto opf updates, no font obfuscation/deobfuscation, no etc...) and then rezipping and using zip file based mod dates to manage things.

Sigil is not just an IDE for editing a set of existing html files (editing a website like most IDE's can do), it does much more than that as it makes automatic changes to the OPF, rebuilds the Nav/NCX, properly updates manifest properties, set semantics, allows editing of metadata in dc format, that most IDE's for xhtml won't do or care about.

You best bet is either to go back to what you do at the xhtml file level, or accept what Sigil does and how it works and use it to its full extent.

Suggestions that try to revert Sigil back to a non-cached, file based system are not going to be accepted. If that bothers you, Sigil might not be for you.

Hope this helps explain why most of your suggestions are not making my To-Do list as you seem to be completely unaware of what Sigil can do in many cases. Sigil has its own way of handling things that either works for you or it doesn't.

Hope this helps,

KevinH

Originally Posted by democrite

Someday some expansion of automation lists, or some revised method of automation could be nice.

Various things I change with all purchased EPUBs. Such takes a bit of time. Some may not be easily automated as of now.

- replace cover w/same image, or set semantics
- remove justification
- if base text isn't 100%/1em, change and scale other styles
- remove font-family from body text styles
- remove line-height from non-headers
- convert to EPUB3 if not yet, change page # tags (e.g. if specified as anchor) to EPUB3, run pagelist plugin
- delete common fonts such as LinLibertine, Charis, etc.
- to make navigation easier for larger books, add to h2+ headers link back to chapter id in html toc
- if no html toc, create - some option to add to NAV/TCX if not there, like hold down alt/option w/menu selection?

possibly more I can't think of.

For some of that, unsure if asked before. Could be some amount of work. Possibly far off as maybe some future feature? As one can modify styles in some web inspector, some style panel where one could easily add or remove attributes to style at cursor point, oh would be so nice.

[KevinH]

@democrite and others who build from source,

FYI, we have recently made changes to master to speed up loading times. You should see the biggest improvement when loading epubs with lots of xhtml files when "clean on open" is set to on (the default).

Since you build your own, you might want to time a few of your largest epub, then rebuild from master and re-time those loading same epubs to see if these improvements will help you.

BeckyEbook has a test epub with over 2500 files that used to take 25 seconds to open is now down to under 5 seconds with the latest master.

Hope this helps ...

Kevin

Originally Posted by democrite

Any EPUB in the hundreds of MB. I am aware of the use of a temp folder. . I was just thinking perhaps - if it is a good idea, I don’t know - updating the EPUB in place with only changes, at times when possible. Or keeping a saved working copy that is updated as such and then replaces original.

Some of us, I’m not sure how many, are more comfortable with somewhat frequent saves.

Whether problematic in some cases, unsure. But would be nice someday if it can be done. If for example, at relevant times on disk EPUB file structure was stored and compared when saving.