format of Saved Searches file

[KevinH]

I will give that a try tomorrow with just the clips file to see what happens.

[BetterRed]

Originally Posted by DiapDealer

Preference continuity on upgrades just isn't something they've ever had to worry about in the past.

That 'ever' doesn't ring true. I've lost keyboard shortcut assignments when I've upgraded. One was F7 for Spellcheck - I've forgotten what grabbed it - might have been Well Formed Check. And that's not the only one I've lost. IIRC it results in two items having the same shortcut.

I can't recall whether I lost them in the most recent upgrade.

Curious - is there some place can I see default keyboard shortcut settings? Apart from to doing a Preferences Reset.

BR

[DiapDealer]

Originally Posted by BetterRed

That 'ever' doesn't ring true.

Sorry, but it certainly rings true to me. I can count on one finger the number of people I've heard reporting lost preferences when doing a normal upgrade. And besides ... it's irrelevant to my point of wanting to avoid it at all costs.

[BKh]

Just wanted to say how grateful I am that you are looking into this.

[Vroni]

Is there a special reason why the saved searches do not remember options and mode or am i doing something wrong? Sometimes i miss especially the options DotAll Minimal Match and Warp in the saved search.

[DiapDealer]

Originally Posted by Vroni

Is there a special reason why the saved searches do not remember options and mode or am i doing something wrong? Sometimes i miss especially the options DotAll Minimal Match and Warp in the saved search.

You're not missing anything. Those settings just aren't part of Sigil's Saved Searches feature. As seen in the Saved Searches dialog. There's a Name (including Group names), the Find criteria, and the Replace criteria.

[Vroni]

Of course i miss the possibility of having different searches being marked as greedy / non greedy or dotall or nondotall

But it is like it is, and i'm not doing anything wrong.

[KevinH]

Okay, using Doitsu's example, I manually edited the sigil_searches.ini file and added a 34th entry and changed the final line size to be 34

Here is a snippet of my new sigil_searches.ini file:

34\Find=Krishna
34\Name=Krishna
34\Replace=K\x1e5b\x1e63\x1e47\x61
4\Find="&(nbsp|#160);(</p>)"
4\Name=Example Searches/Remove Non-Breaking Spaces/Trailing space
4\Replace=\\2
5\Find="(<p[^>]*>)&(nbsp|#160);"
5\Name=Example Searches/Remove Non-Breaking Spaces/Leading space
5\Replace=\\1
6\Find=\x2002
6\Name=Example Searches/Convert Characters to Entities/ensp
6\Replace="&ensp;"
7\Find=\x2003
7\Name=Example Searches/Convert Characters to Entities/emsp
7\Replace="&emsp;"
8\Find=\x2009
8\Name=Example Searches/Convert Characters to Entities/thin space
8\Replace="&thinsp;"
9\Find=\xad
9\Name=Example Searches/Convert Characters to Entities/shy
9\Replace="­"
size=34

Next I edited Sigil/src/MiscEditors/SearchEditorModel.cpp and added a call to use setIniCodec immediately after creating the settings instance as follows:

void SearchEditorModel::LoadData(const QString &filename, QStandardItem *item)
{
    SettingsStore *settings;

    if (filename.isEmpty()) {
        settings = new SettingsStore(m_SettingsPath);
    } else {
        settings = new SettingsStore(filename);
    }

    settings->setIniCodec("UTF-8");

    int size = settings->beginReadArray(SETTINGS_GROUP);

Then I rebuilt Sigil and fired it up and added the following line to my html file:

<p> this is a test: Krishna</p>

Next I went to Tools->Saved Searches and chose Krishna and ran the find and replace on it and received the correct result:

Kṛṣṇa

So, at least for Saved Searches Find and Replace, setting the ini Codec to utf-8 did the right thing. I am not sure how that would work with general "clips" so we will need to check it as well.

[DiapDealer]

Originally Posted by Vroni

Of course i miss the possibility of having different searches being marked as greedy / non greedy or dotall or nondotall

But it is like it is, and i'm not doing anything wrong.

Also remember that most of those options: dotall, un/greedy can be specified (or reversed) in the regular expression itself. I always leave the the checkbox gui options in the same exact same state, and use the expression itself to set greed/lazy multi-line, case insensitivity, etc..., for each particular search.

Also note that if you always leave "All Files" selected you can override/force search to operate only on the current file by holding the Ctrl key down while clicking on the action.

I never uncheck "wrap," myself *shrug*

[DiapDealer]

Originally Posted by KevinH

Next I went to Tools->Saved Searches and chose Krishna and ran the find and replace on it and received the correct result:

Kṛṣṇa

So, at least for Saved Searches Find and Replace, setting the ini Codec to utf-8 did the right thing. I am not sure how that would work with general "clips" so we will need to check it as well.

What's it do to the saved searches ini file itself when making changes to Saved Searches? Will it keep existing hex as hex and new utf-8 as utf-8 in a mixed mode, or will the hex encoded entries be changed to utf-8 when saved with the new ini-encoding? I can live with the former, but the latter would be great!

[KevinH]

Next I edited sigil_clips.ini and changed the 22nd entry to:

22\Name=Krishna
22\Text=K\x1e5b\x1e63\x1e47\x61

Then edited Sigil/src/MiscEditors/ClipEditorModel.cpp and added the call to set the ini codec to utf-8:

void ClipEditorModel::LoadData(const QString &filename, QStandardItem *item)
{
    SettingsStore *settings;

    if (filename.isEmpty()) {
	settings = new SettingsStore(m_SettingsPath);
    } else {
        settings = new SettingsStore(filename);
    }

    settings->setIniCodec("UTF-8");

    int size = settings->beginReadArray(SETTINGS_GROUP);

Then fired up Sigil and opened the Clips view window and selected Krishna and it properly pasted in:

Kṛṣṇa

And then I used the clips editor to create and save a new Krishna2 clip and exited out of Sigil and went and looked at the sigil_clips.ini file and saw the following:

22\Name=Krishna
22\Text=K\x1e5b\x1e63\x1e47\x61
23\Name=Krishna2
23\Text=K\x1e5b\x1e63\x1e47\x61

So setting the codec of the ini file to utf-8 was NOT enough to get it to save the ini file as a utf-8 encoded file.

So so all of this testing may be in vain. I will take a closer look at the code that loads data and saves data in these models to see if I can see what is going on.

KevinH

[KevinH]

Ahh! So I missed adding a call to setIniCodec in the SaveData() routine in ClipEditorModel.cpp whcih when I added it and recompiled then open and saved the clips, I now see the following in sigil-clips.ini

20\Name=Example Clips/Miscellaneous/lowercase
20\Text=\\L\\1\\E
21\Name=Example Clips/Miscellaneous/Titlecase
21\Text=\\u\\1
22\Name=Krishna
22\Text=Kṛṣṇa
23\Name=Krishna2
23\Text=Kṛṣṇa

So it looks like it works just fine if we use setIniCodec to utf-8 both in LoadData() and SaveData() immedately after the SettingsStore object in created. It nicely groks hex escapes on input but saves them as utf-8.

So it appears we can easily convert clips and searches ini files to use utf-8 with no consequences at least if the users do not downgrade Sigil after the change.

What do you think?

KevinH

[KevinH]

BTW, you have to make the same change to add setIniCodec to utf-8 in the SaveData() routine in Sigil/src/MiscEditors/SearchEditorModel.cpp to get things to work properly in Find and Replace ini files too.

Of course, if we decide to make this change, we can always just change Sigil/src/Misc/SettingsStore.cpp and add the call to setIniCodec in the constructor to make sure all instances of SettingsStore properly set the ini codec, that way we will not miss anyplace, and we do not have to hunt to find everyplace where SettingsStore is used.

Thoughts?

KevinH

[KevinH]

In case anyone wants to test this in their own current build of Sigil just to verify it does not mess anything up for them:

1. Make backup copies of all your sigil related ini files in your Sigil Preferences folder

You can use the Go To Sigil Preferences Location button or use your file explorer:

    For Example: on a Mac they can be found here:
        /Users/YOURUSERID/Library/Application Support/sigil-ebook/sigil
            sigil.ini
            sigil_index.ini
            sigil_clips.ini
            sigil_searches.ini

2. Make the following changes to Sigil/src/Misc/SettingsStore.cpp

--- src/Misc/SettingsStore.cpp~	2017-12-05 09:01:06.000000000 -0500
+++ src/Misc/SettingsStore.cpp	2018-10-23 10:59:11.000000000 -0400
@@ -100,11 +100,13 @@
 SettingsStore::SettingsStore()
     : QSettings(Utility::DefinePrefsDir() + "/sigil.ini", QSettings::IniFormat)
 {
+     setIniCodec("UTF-8");
 }
 
 SettingsStore::SettingsStore(QString filename)
     : QSettings(filename, QSettings::IniFormat)
 {
+     setIniCodec("UTF-8");
 }
 
 QString SettingsStore::uiLanguage()

Then rebuild and launch Sigil. Sigil should now safely convert all of your ini files to use utf-8 format.

Then test everything and report back if any clips, preferences, saved searches, etc cause you any issues.

Please note - this is a one-way change to your ini files to make them more readable and editable by humans. Once you are done testing, if you want to revert back to the official build and then you will need to put back your sigil related ini files from your backups **before** firing up the older version of Sigil.

[DiapDealer]

Originally Posted by KevinH

BTW, you have to make the same change to add setIniCodec to utf-8 in the SaveData() routine in Sigil/src/MiscEditors/SearchEditorModel.cpp to get things to work properly in Find and Replace ini files too.

Of course, if we decide to make this change, we can always just change Sigil/src/Misc/SettingsStore.cpp and add the call to setIniCodec in the constructor to make sure all instances of SettingsStore properly set the ini codec, that way we will not miss anyplace, and we do not have to hunt to find everyplace where SettingsStore is used.

Thoughts?

KevinH

Unless we can envision a situation where individual config files might need to be something other than utf-8, I like the idea of the latter, one-size-fits-all approach, myself.

I'll try to do some testing on Linux and Windows soon.