Suggestion for Improving the "Insert Special Character" Feature

[BeckyEbook]

I have another proposal that might fit the goal of keeping the existing feature small while making it more useful for users who want customization.

What if Sigil optionally looked for a characters.txt file in the Sigil preferences folder?

If the file does not exist, nothing changes at all: Sigil uses the current built-in list exactly as it does now.

If characters.txt exists, its contents replace the built-in list. This would let users create their own small list of frequently used characters without adding another preference or increasing Sigil's built-in data.

The format could be deliberately simple: one entry per line, with up to four "|"-separated fields:

character|display text|entity|description

The fields could also be progressively optional, so simply writing:

←

would be enough, while something like:

←|sl|←|left arrow

could provide a different button label and tooltip.

For the custom list, I would also suggest a simple flow layout rather than the current fixed-column grids. The buttons would naturally wrap according to the dialog width, with the existing scroll area handling larger lists. This works particularly well when custom entries have different display widths. [* – That's just my personal opinion]

The important part is that this would be completely opt-in. Users who are happy with the current list and layout would never see any difference, while users who want a personal set of characters could create one very simple text file.

It also seems less intrusive than putting the customization directly into sigil_v6.ini: editing or deleting a standalone text file is straightforward, and the normal Sigil configuration remains untouched.

In the screenshots, I'm showing a set of characters planned by @byword + original with fixed-column grids.

[KevinH]

I like that idea!

But the encoding of the special_characters.txt file would be crucial. No utf-16 and bom, just pure utf-8 with no bom. There is also the problem of copying and pasting into the special_characters.txt file from other apps messing up the encoding, changing line ends, no newline in list line, etc.

In the worst case we could replace the character field with 2 to 6 hex chars, followed by a name, then self generate the actual character from it. You could use the same mechanism we use now to show a description if the character under the cursor in CV to generate a name description. You could also generate a numeric entity instead of a named entity from that code. Named entities require a special doctype under epub2 and except for the xml basic ones they are illegal in epub3, so their usefulness are about done.

I guess as an alternative we could create a special_characters.xml file that would have a clear utf-8 header, and that would come with better builtin parsing for more consistency.

I just fear that encoding things in text files under Windows text editors especially is going to be an issue for many users.

[KevinH]

If we go the xml route, then a very simple syntax like:

<specialchars>
<char hex="2021" label="a short name" />
...
</specialchars>

Might be useful, as we could use our existing xml tools to validate it is wellformed, parse it and build the table. No added tools needed for parsing.

And then we can have Sigil create this file from its current Insert Special Characters set if it does not already exist and we can unify it to a single editable approach.

[BeckyEbook]

I just fear that encoding things in text files under Windows text editors especially is going to be an issue for many users.

Unfortunately
During testing, I simply created a text file, but I was sure it was UTF-8 and prepared it with full awareness of that.

XML is also a good idea.
And eventually, a SpecialCharsEditor plugin might be developed that will allow you to edit this file.

[KevinH]

I like the xml approach. I will take a shot at implementing that so people can try it out.

[KevinH]

Hi BeckyEBook,
Would you please post a link or point to your SelectCharacter routine so I can see how you are doing the free form layout and how it ties to the signal mapper?

Thanks,

Kevin

[KevinH]

Hi BeckyEBook,

Wow! Nicely done. It seems to work well with or without the characters.txt file in my Sigil Prefs.

A nitpick:

- The flow layout certainly holds more chars but the structure of chars is lost. In the original, we started with spaces as the first group, then the main set of html named entities in the next group, followed by Greek chars in the third group, and finally math symbols in the fourth group.

That structure made it much easier for me to find characters but it got lost when using characters.txt.


If we decide to go with an xml structure, perhaps we could add a group tag.

Something like:

<specialchars>
<chargroup label="Html Named Entities">
<char hex="2021" label="a short name" />
...
</chargroup>
...
</specialchars>

Then create a divider the forces things to a new line with that label somehow in the FlowLayout.

We should also provide a filter feature of some sort that filters the descriptions.

Thoughts?

[BeckyEbook]

I admit I thought about groups, but I do a lot of things by trial and error, so in this case I mainly wanted to test the idea itself.

Custom groups, custom characters within groups… That's fine with me.
Then, even if the characters overflowed when resizing the window, the very existence of groups would introduce a kind of "order," and the user wouldn’t have to search through a huge mass of symbols with their eyes – they could simply locate a group and select a specific character.

As you mention, this unnamed groups exists (spaces, main, Greek, math) by default and is clearly visible, but introducing a default "flow" instead of "grid" would, IMHO, also be beneficial (see my earlier second screenshot in post #31 – with a large window, there’s empty space on the right; with a small one, a horizontal scroll bar appears).

[KevinH]

Agreed. We just have to figure out out to create a line break with a label header and terminating line break that is not clickable in the Flow layout.

Let's both think about this. Perhaps we just go with what you have now and then refine it for a future release if needed. But I really like the xml approach with groups too.

Hmm...

[BeckyEbook]

XML is definitely a good idea.

Of course, if you really wanted to, you could add [GREEK], [MATH] groups to a text file as well, but I think XML would be a more Sigil-like solution here, and the structure with groups practically begs for XML.

[KevinH]

I found the official Qt6 flowlayout example code and figured out how to modify it to force line breaks with a empty widget with a special objectName.

So I will split out FlowLayout (with the added line break capability) into its own Widgets/class in case it is ever needed someplace else.

Then I will modify your code to look for chars whose string is [GroupName] and special case it to use the GroupName as a QLabel with forced line breaks on both sides.

Once I get that working, I will migrate everything to xml and use special_characters.xml.

I will let you know when I get something working for test purposes.

Thanks!

[BeckyEbook]

Great news!

In the meantime, I'm already working on a plugin that will let you select the appropriate groups/characters to create your own configuration.
Groups and subgroups based on Calibre's excellent and logical categorization [link], rather than the default Unicode group layout [link].

Please note the top-level group "Favorites," which will appear first, at the very top, in the "Insert Special Character" window. Characters in the "Favorites" group may or may not appear later in another group.
Please keep this in mind when designing the XML structure.
The groups themselves will be the names of the subgroups from that window (like "Armenian", "Georgian", and not "European scripts").

P.S. Of course, this is just a beta version that neither reads nor writes XML, since I don't know the final structure yet

[KevinH]

Wow, that will be great! I hope to have something to play with by tomorrow late. I will push it to my personal Sigil repo until all the kinks are worked out.

All group headings will allow any string text so that should work. That said, I do *not* plan to support nested group tags in the xml version. So one group tag can not hold another but each group can hold any number of chars.

And any char can be repeated as many times as you want since many natural groupings overlap.