format of Saved Searches file

[BKh]

I have about 150 search and replace pairs and I'm trying to put them into the sigil_searches.ini file manually. Everything is working out fine, except the replace field includes unicode characters (e.g. āīūḷṇṭḍṁṅ) and they are converting to some sort of hex codes. That wouldn't be a problem, except that they aren't converting correctly. For example ū becomes \xc5 in the file and then shows up as Å« in the saved search dialogue.

Sorry if it seems I don't know what I'm talking about. Because I don't

I hope someone can tell me what I might be doing wrong.

[theducks]

[stealing your thread] I would like to go the other way Sigil -> Calibre

A Import / Export plugin please

[BKh]

Originally Posted by theducks

[stealing your thread] I would like to go the other way Sigil -> Calibre

A Import / Export plugin please

Steal away... But I don't understand what you mean. Perhaps my OP wasn't clear. What does this have to do with calibre? Or are you talking about having a way to export saved searches into Calibre. Pardon my ignorance. I didn't even know that Calibre had a saved search feature.

[theducks]

Originally Posted by BKh

Steal away... But I don't understand what you mean. Perhaps my OP wasn't clear. What does this have to do with calibre? Or are you talking about having a way to export saved searches into Calibre. Pardon my ignorance. I didn't even know that Calibre had a saved search feature.

Precisely! I started with Sigil 0.99 so I have quite a catalog of S&R templates.

I use Sigil for my base edits, then switch to Calibre for debug, clean the CSS.
Its a toss, for using the toolbag

[Tex2002ans]

Originally Posted by BKh

That wouldn't be a problem, except that they aren't converting correctly. For example ū becomes \xc5 in the file and then shows up as Å« in the saved search dialogue.

Hmmm, I'll have to poke around in the Saved Searches ini again. I haven't looked in a while (very rarely have to transfer these files over, and rarely use accented characters in them).

The thing I do recall was there being a bug with line endings (?) between the Windows/Mac versions... but that was years and years ago (may have been fixed by now).

Originally Posted by BKh

Steal away... But I don't understand what you mean. Perhaps my OP wasn't clear. What does this have to do with calibre? Or are you talking about having a way to export saved searches into Calibre. Pardon my ignorance.

Yes, that's what theducks meant. He just wishes there was a way for you to be able to Import/Export Saved Searches between Sigil<->Calibre too.

Originally Posted by BKh

I didn't even know that Calibre had a saved search feature.

Yep, it's under View > Saved Searches.

When the Calibre Editor was created, Kovid did a fantastic job of porting pretty much every functionality from Sigil over.

[Doitsu]

Originally Posted by BKh

I have about 150 search and replace pairs and I'm trying to put them into the sigil_searches.ini file manually.

You'll need to replace all accented characters with hexadecimal Unicode codes:

ā \x101
ī \x12b
ū \x16b
ḷ \x1e37
ṇ \x1e47

For example to replace Krishna with Kṛṣṇa you'd use:

1\Name=Krishna
1\Find=Krishna
1\Replace=K\x1e5b\x1e63\x1e47\x61

(For some odd reason the last "a" also needed to be encoded as a hexadecimal character. Maybe @KevinH or @DiapDealer can shed some light on this.)

Originally Posted by theducks

A Import / Export plugin please

The problem with this idea is that Calibre saves additional information with each entry (case_sensitive, mode, dot_all) that would be lost if you converted it to the Sigil Saved Searches format.

For example, the above search looks like this in tweak_book_gui.json:

{
  "searches": [
    {
      "case_sensitive": false, 
      "dot_all": false, 
      "find": "Krishna", 
      "mode": "normal", 
      "name": "Krishna", 
      "replace": "Kṛṣṇa"
    }
  ], 
  "version": 1}

Importing Sigil seaved searches into Calibre Editor should be relatively easy for a Calibre programmer. @kovidgoyal might even add this as a feature to Calibre Editor if enough users ask for it.

[BKh]

Originally Posted by Doitsu

You'll need to replace all accented characters with hexadecimal Unicode codes:

ā \x101
ī \x12b
ū \x16b
ḷ \x1e37
ṇ \x1e47

For example to replace Krishna with Kṛṣṇa you'd use:

1\Name=Krishna
1\Find=Krishna
1\Replace=K\x1e5b\x1e63\x1e47\x61

(For some odd reason the last "a" also needed to be encoded as a hexadecimal character. Maybe @KevinH or @DiapDealer can shed some light on this.)

So, I think there is an issue with using two different forms of the hex codes. In your example, beautiful as it is, it is a solid string of hex codes. without being followed by anoter hex code, how does the software know that some of the hex are 3 digit and some are 4?

Is there some way to wrap the codes so it is clear where they start and stop?

Again, sorry if I am using wrong terminology.

[DNSB]

Originally Posted by BKh

So, I think there is an issue with using two different forms of the hex codes. In your example, beautiful as it is, it is a solid string of hex codes. without being followed by anoter hex code, how does the software know that some of the hex are 3 digit and some are 4?

Is there some way to wrap the codes so it is clear where they start and stop?

Again, sorry if I am using wrong terminology.

1\Replace=K\x1e5b\x1e63\x1e47\x61

The \x starts the hex code string. A non-hex character ends it. In the sample used, since a is a valid hex character, it had to be replaced by it's equivalent hex representation or the the last character would have been seen as \x1e47a and likely treated as an invalid character. Leading zeros are not normally typed so \x61 instead of \x0061. The copyright sign (©) as \xa9 instead \x00a9 or a capital A macron (Ā) as \x100 instead of \x0100.

Hmmm... time to quit since my eyes are glazing over.

[DiapDealer]

@DNSB is correct. There's not "two different" forms of hex codes being used.

As far as manually creating these files ... more power to you, but that wasn't really taken into consideration when the Saved Searches feature was first implemented. The actual creation of the file is handled by Qt in such a way that the same code will work in cross-platform situations. But since Qt automatically handles things like line endings, it's not likely that a file created on one platform will always be 100% compatible with Sigil on another platform.

I will look to see how Qt creates the file, though. Perhaps something in python could be done to change unicode into the hex representation that Sigil expects the file to be in.

[KevinH]

This link might be relevant:

https://stackoverflow.com/questions/...e-unicode-well

So we could set the encoding for the ini file to be utf-8, and as long as it still groks the ascii with hex codes on input, we should be okay to use that. It will definitely make the ini more user editable.

[DiapDealer]

Originally Posted by KevinH

and as long as it still groks the ascii with hex codes on input, we should be okay to use that.

That's the sticking point as I see it. I'm all for it, as long as it doesn't invalidate preexisting files with hex codes. Are you talking about only making the change for this particular preference file (to be safe)?

[KevinH]

I was kind of thinking about doing that for all ini files again as long as it still can grok the ascii/hex code on input. I just like users to be able to read, understand, and even edit all of their Sigil ini files if they so desire.

Alternatively we could all pretty easily create a short python app to convert all Sigil ini files to be utf-8 encoded if needed (a one-time tool) or even add code to Sigil itself if need be to do the conversion.

What are your thoughts?

[DiapDealer]

I'm not opposed at all to more human-readable preference files. I just don't want users to have to manually do anything special before installing the latest Sigil update to safeguard their current preferences. Preference continuity on upgrades just isn't something they've ever had to worry about in the past.

[KevinH]

Agreed. The first thing to check is if an ini file is set to be utf-8 encoded, can it still handle hex escapes.

[DiapDealer]

Originally Posted by KevinH

Agreed. The first thing to check is if an ini file is set to be utf-8 encoded, can it still handle hex escapes.

True. We may have nothing to worry about.