Ideas for New Improvements or Features

[shopma2]

I would like a keyboard-shortcut in the Spellchecker for 'Change Selected Word to:'. Maybe it already exists, but I haven't been able to find it.
Thanks.

[BeckyEbook]

Originally Posted by BetterRed

Make Search->File & Replace (and its shortcut) a Show and Hide toggle.

I will repeat my comment from last year.

I didn't want you to think there was any conspiracy against the idea. It's not.

I took special interest in the idea again:
1. installed the dangerous version 1.4.0
and
2. built fresh Sigil 2.4.2 with a fix for toggle the F&R window.

By the way, I understood (again!) why this solution was rolled back in version 1.4.1.

I already explain.
Currently, we have such a situation. You are sitting in Code View and editing text. Hands-on the keyboard, so you invoke the F&R window via the Ctrl-F shortcut (or another one if you have changed the default settings). Whether the window is visible or not – focus jumps to F&R, you enter something into the "Find" field, then tab and complete the "Replace" field. Later "Replace" or something else. Everything works great.

After implementing the toggle fix (show/hide):
What happens when the F&R box is visible on the screen, and you want to jump to it via Ctrl-F? The F&R window will disappear! You have to apply the keyboard shortcut a SECOND time to make it appear again! This is what kept me awake at night when this switching was implemented in version 1.4.0. Fortunately, version 1.4.1 came out right away, which had other problems and was soon replaced by 1.4.2.

If you have an idea for a solution to this problem – I am really in favor of consistency in switching all windows.

[KevinH]

For those following the suggestion for a python function replace capability, I have grabbed current RegexFunctionReplace plugin and have updated it to PySide6 and added some code to do Python Syntax Highlighting, fixed autosave, run it to test. And it all works again.

To expand it to allow all other Regex flags would be easily done (although those flags canbe set by codes in the Find Field), and to add the ability to do just a find and later replace in just the current file is doable as well.

It does function well. But there are no simple pre-wriiten routines provided, which we could add as well.

I then studied how to possibly integrate it into Sigil itself but there are a few issues that create problems:

ReplaceAll in Sigil is stateless and always work starting with the current file so numbering would be an issue.

Internally we use Qt's QString which is a sequence of utf-16le values, And pcre2 internally works on those. The issue is a match result from pcre2 internal to Sigil uses offsets in the text to each start and end positions within that text all based on positions in utf-16le sequences (ie in QChars).

But Python str type used utf-32le values for each char which means any position offset values provided by Sigil's pcre2 could/would be invalid when provided to Python.

So each PCRE2 match result would have to be properly translated and rebuilt to be used inside Python.

It is not that all of this is insurmountable, but it would require a lot of work. So I really need to see that the revised RegexFunctionReplace plugin can NOT be made to do the job, and that this is something that would be used often by many Sigil users.

[BetterRed]

Originally Posted by BeckyEbook

I will repeat my comment from last year.

I didn't want you to think there was any conspiracy against the idea. It's not.

I took special interest in the idea again:
1. installed the dangerous version 1.4.0
and
2. built fresh Sigil 2.4.2 with a fix for toggle the F&R window.

By the way, I understood (again!) why this solution was rolled back in version 1.4.1.

I already explain.
Currently, we have such a situation. You are sitting in Code View and editing text. Hands-on the keyboard, so you invoke the F&R window via the Ctrl-F shortcut (or another one if you have changed the default settings). Whether the window is visible or not – focus jumps to F&R, you enter something into the "Find" field, then tab and complete the "Replace" field. Later "Replace" or something else. Everything works great.

After implementing the toggle fix (show/hide):
What happens when the F&R box is visible on the screen, and you want to jump to it via Ctrl-F? The F&R window will disappear! You have to apply the keyboard shortcut a SECOND time to make it appear again! This is what kept me awake at night when this switching was implemented in version 1.4.0. Fortunately, version 1.4.1 came out right away, which had other problems and was soon replaced by 1.4.2.

If you have an idea for a solution to this problem – I am really in favor of consistency in switching all windows.

Should be like other optional windows, a menu item/shortcut option to toggle show and hide, and a menu item/shortcut option to give it keyboard focus. The fact that it is not dockable should be irrelevant.

Pretty sure I already suggested a solution (may have been for calibre which has the same (IMO) defect): an additional menu item/shortcut (with no default) option to close the F&R window. I can't recall why that was rejected.

BR

[KevinH]

For those interested in a Python Function replace capability:

I have continued to update the RegexFunctionReplace to work with current Sigil.

I have attached a screen cap which show it run a titlecase function on the contents of every h1 tag as a simple example:

Find: <h1\s[^>]*>([^<>]*)</h1>

# titlecase replacement python function

def replace(match, number, file_name, metadata, data):

	def titlecaseit(text):
		textlow = text.lower()
		result=''
		prevc = None
		for i, c  in enumerate(textlow):
			if c.isspace():
				result = result + c
			else:
				if i == 0 or prevc.isspace():
					result = result + c.upper()
				else:
					result = result + c
			prevc = c
		return result

	if match is None:
		return ''

	if match:
		sp = match.start(1) - match.start(0)
		ep = match.end(1) - match.start(0)
		rv = match.group(0)
		newtitle = titlecaseit(match.group(1))
		rv = rv[0:sp] + newtitle + rv[ep:]
		return rv
	return match.group(0)

This is a duplicate of the simple TitleCase code used by Sigil's own TitleCase routine just written in python. Please note this is meant as a simple demonstration function.

That program was run on the Alice in Wonderland epub where one of the xhtml files had been modified to have an h1 tag with "contents" in it.

That h1 tag was found and "fixed".

So I really need to see some examples of things that people are really using python function replacement for so that I know what additional capabilities are actually needed.

So @Turtle91 and others what functions do you actually need and use?

I am still figuring out if and how this might be3 integrated into Sigil itself instead of a plugin, but updating the plugin would certainly help.

If anyone wants a copy of the updated plugin code, please let me know.

FYI: I still need to handle dark mode vs light mode and highlight colours for the updated RegexFunctionReplace plugin as settings and way to see if dark or light when launched.

[nabsltd]

Originally Posted by KevinH

As for Python Function Replace - outside of numbering something or doing Title Case (or some other case conversion) how many people have actually used their own python function replace routines?

The only time I use Calibre's function S&R is for case conversion, since it's one thing that a plain regex just can't do. Since I'm a pretty heavy editor, and do a lot of complex search and replace, I suspect that is true of most everyone else, too.

[KevinH]

Originally Posted by nabsltd

The only time I use Calibre's function S&R is for case conversion, since it's one thing that a plain regex just can't do. Since I'm a pretty heavy editor, and do a lot of complex search and replace, I suspect that is true of most everyone else, too.

This is not done using the Find button and then the replace button with the case change flags operating on capture groups, right? ie. these used in the replace field before and after capture groups \1 \2 etc.

    // Case changes can be:
    // \l Lower case next character.
    // \u Upper case next character.
    // \L Lower case until \E.
    // \U Upper case until \E.
    // \E End case modification.
    // * Note: case changes cannot stop within a segment. Meaning
    // a \L within a \U will be ignored and the \U will be honored until

Or by hitting the Case Change tool bar icon after hitting the Find Button?


This is done by using Replace All button, correct?

And exactly what case conversions do you need:

Title Case
Upper Case
Lower Case
Capitalize

Any others?

And how smart is the calibre titlecase conversion?

Titlecasing can be actually quite hard to get right depending on what styles are common in which languages. For example: In English should you capitalize little words such as by, of, on, in, etc when not found at the start of a line? Some say no others yes. And what about Roman numerals and abbreviations in a title as they must be specially handled (ie. there can by combinations of Roman Numerals that make words such as MIX. You can find whole language dependent python modules on github that can still make mistakes.

So really well done title casing truly needs to be done by using Find to detect a potential title and looking at each potential title text word by word and then fixing it by hand (IMHO) using your style conventions or that of the editor or publisher.

Update:

Even the most complete version of titlecase that I could find for python with a license we could use in Sigil:

https://github.com/ppannuto/python-t...se/__init__.py

Has issues with many things:

https://github.com/ppannuto/python-titlecase/issues/96


And this is all just issues in English. With other languages, the results would be much worse.

[KevinH]

Hi All,

Okay, I borrowed a bunch of the case change code from calibre and adapted it to work in our Sigil plugin environment. It is ready for testing and feedback.

I have made the following changes since v0.1.0

• updated to use PySide6 from PyQt5
• added a working PythonSyntaxHighlighter
• modified to better indicate replace all progress and completion
• modified editor to use QPlainTextEdit instead of QTextEdit
• modified editor to set tab stops to 4 spaces
• handle dark vs light syntax coloring changes based on Palette.Change events
• now implements the following case change functions (built-in)
  • uppercase
  • lowercase
  • capitalize
  • titlecase
  • swapcase
  • uppercase_ignore_tags
  • lowercase_ignore_tags
  • capitalize_ignore_tags
  • titlecase_ignore_tags
  • swapcase_ignore_tags
• All borrowed from calibre and modified to work without icu and in our plugin framework)

Note: Just as was done in the original version, the gui used by this plugin autosaves any new functions you create and once it has completed you simply close the gui to exit


Note: There is no gui to set regular expression flags yet but you can set them right at the start of the Find string as follows:

(?aiLmsu)

• a: use ASCII instead of the default Unicode
• i: ignore case
• L: locale dependent
• m: multi line
• s: dotall (dot matches all)
• u: use Unicode (the default)

For example to search a multi-line source and set dotall in a regular expression to find h1 through h3 tags that have no nested tags in them (just text) you could use the following Find:

(?sm)<h[123]\s[^>]*>([^<>]*)</h[123]>

and of course you can patterns to be non-greedy as normal:
ie. make *?, +?, ??, or {m,n}? [...] match as little text as possible.


So anyone who has used calibre's python function replace to work with case changes, please give this a try and report back any successes and or issues.

It should be very very close to what calibre provides at least for these case change functions.

[Turtle91]

Sorry I haven’t replied quickly… was a little busy with my daughter getting married

I’ll play around with it a bit and see how it goes. Thanks!!

[KevinH]

Originally Posted by BetterRed

Should be like other optional windows, a menu item/shortcut option to toggle show and hide, and a menu item/shortcut option to give it keyboard focus. The fact that it is not dockable should be irrelevant.

Pretty sure I already suggested a solution (may have been for calibre which has the same (IMO) defect): an additional menu item/shortcut (with no default) option to close the F&R window. I can't recall why that was rejected.

BR

Okay I took a look at this and there already was a m_FindReplace->HideFindReplace() routine that exists in MainWindow.


Update:

Okay I have added this to Sigil master, and this new action will appear in the next release of Sigil


I will look into adding a QAction to main.ui and registering it but assigning no default shortcut so that a user can assign a shortcut if needed to close F&R

[KevinH]

Originally Posted by shopma2

I would like a keyboard-shortcut in the Spellchecker for 'Change Selected Word to:'. Maybe it already exists, but I haven't been able to find it.
Thanks.

Okay I looked into this. There is no shortcut direct to that button.

That said, that whole SpellcheckEditor dialog is meant to be traversed completely from your keyboard using the Tab key and the Shit-Tab key to rapidly change focus to whatever you want.

For example, I hit the down arrow and focus goes immediately to the Word list then hit Tab 3 times and Focus in now on the Change Selected Word To button and one more Tab will go to the text field in case I want to change the work to use as replacement and Shift Tab will bring me back to that button.

This way to change focus works through many dialogs all throughout Sigil and in many other programs.

Your fingers never have to leave the keyboard.

This Tab and Shift Tab to move focus around works by default on Windows and Linux.

On MacOS
You actually have to enable it in MacOS system settings as follows *before* launching Sigil (or any other program that uses Tab in that way)
- open "System Settings" and select "Keyboard"
- make sure "Keyboard navigation" is enabled.

I enabled this on my Macs years ago and never looked back. Very useful for Qt based programs.

So I do not think yet another shortcut is really needed for those who are heavy users of the keyboard in this dialog. Just try using the Tab and Shift Tab to move around and do almost everything in that dialog.

[KevinH]

Do we need a new Tool menu item to "Use Standard File Extensions"?

This would be a tool which would walk all files in the epub, and based on their declared OPF manifest media-type check the file extension and if needed add or change the extension.

Basically it would be an reverse (with more alternate media-types added) of this kind of table:

void MediaTypes::SetExtToMTypeMap()
{
    if (!m_ExtToMType.isEmpty()) {
        return;
    }
    // default to using the preferred media-types ffrom the epub 3.2 spec
    // https://www.w3.org/publishing/epub3/epub-spec.html#sec-cmt-supported
    m_ExtToMType[ "bm"    ] = "image/bmp";
    m_ExtToMType[ "bmp"   ] = "image/bmp";
    m_ExtToMType[ "css"   ] = "text/css";
    m_ExtToMType[ "epub"  ] = "application/epub+zip";
    m_ExtToMType[ "gif"   ] = "image/gif";
    m_ExtToMType[ "htm"   ] = "application/xhtml+xml";
    m_ExtToMType[ "html"  ] = "application/xhtml+xml";
    m_ExtToMType[ "jpeg"  ] = "image/jpeg";
    m_ExtToMType[ "jpg"   ] = "image/jpeg";
    m_ExtToMType[ "js"    ] = "application/javascript";
    m_ExtToMType[ "m4a"   ] = "audio/mp4";
    m_ExtToMType[ "m4v"   ] = "video/mp4";
    m_ExtToMType[ "mp3"   ] = "audio/mpeg";
    m_ExtToMType[ "mp4"   ] = "video/mp4";
    m_ExtToMType[ "ncx"   ] = "application/x-dtbncx+xml";
    m_ExtToMType[ "oga"   ] = "audio/ogg";
    m_ExtToMType[ "ogg"   ] = "audio/ogg";
    m_ExtToMType[ "ogv"   ] = "video/ogg";
    m_ExtToMType[ "opf"   ] = "application/oebps-package+xml";
    m_ExtToMType[ "opus"  ] = "audio/opus";
    m_ExtToMType[ "otf"   ] = "font/otf";
    m_ExtToMType[ "pls"   ] = "application/pls+xml";
    m_ExtToMType[ "pdf"   ] = "application/pdf";
    m_ExtToMType[ "png"   ] = "image/png";
    m_ExtToMType[ "smil"  ] = "application/smil+xml";
    m_ExtToMType[ "svg"   ] = "image/svg+xml";
    m_ExtToMType[ "tif"   ] = "image/tiff";
    m_ExtToMType[ "tiff"  ] = "image/tiff";
    m_ExtToMType[ "ttc"   ] = "font/collection";
    m_ExtToMType[ "ttf"   ] = "font/ttf";
    m_ExtToMType[ "ttml"  ] = "application/ttml+xml";
    m_ExtToMType[ "txt"   ] = "text/plain";
    m_ExtToMType[ "vtt"   ] = "text/vtt";
    m_ExtToMType[ "webm"  ] = "video/webm";
    m_ExtToMType[ "webp"  ] = "image/webp";
    m_ExtToMType[ "woff"  ] = "font/woff";
    m_ExtToMType[ "woff2" ] = "font/woff2";
    m_ExtToMType[ "xhtml" ] = "application/xhtml+xml";
    m_ExtToMType[ "xml"   ] = "application/xml";
    m_ExtToMType[ "xpgt"  ] = "application/vnd.adobe-page-template+xml";
}

where instead of using the file extension to look up a media-type, you would use the media-type to look for the proper file extension and use that in a rename if needed.

This could be done easily and make sure all css files end with .css, all xhtml files end with .xhtml, all jpegs files end up with .jpg, etc.

Could be a nice first step when you first start editing a rough built epub.

Would this kind of tool be useful to anyone?

If so I could also add it as a possible Automate action as well.

Any thoughts?

[DNSB]

I've had to manually cleanup several messed up epubs. While I wouldn't use it that often, it would be useful on occasion.

[KevinH]

Originally Posted by DNSB

I've had to manually cleanup several messed up epubs. While I wouldn't use it that often, it would be useful on occasion.

Noted: - "used but not often". Thank you for your feedback.

Let's wait to hear from a few others before making a final decision yea or nay.

[BeckyEbook]

I like this idea because I like consistency.
But will the user have a choice?
E.g., whether the mime type “image/jpeg” will change the extension to “.jpg” or “.jpeg”?
Users have their preferences.
I always choose “.jpg”, but someone else may prefer “.jpeg”.