How to write regex function which uses dictionary?

[Phssthpok]

I have a book which has hyphens instead of em dashes, and I'm trying to fix it. Using a regex like "-(and|but|with)" catches a few cases, but it would be better to find all "\w+-\w+" which are not in the current dictionary, which would catch about 99% of all cases (and leave the remainder to the proofreading stage).

How could I write a regex function to do this?

[kovidgoyal]

https://manual.calibre-ebook.com/fun...phenated-words

[Phssthpok]

Originally Posted by kovidgoyal

https://manual.calibre-ebook.com/fun...phenated-words

Aha! Thank you!

[Phssthpok]

Originally Posted by Phssthpok

Aha! Thank you!

Hmm, having done this I realise that what I really need is a way to *find* things which are not in the dictionary, not all hyphenated words -- since there gazillions of those to wade through, and far fewer which are non-dictionary items.

Seems to be no way to do this. Plugin maybe? Back to the drawing board...

[lomkiri]

Originally Posted by Phssthpok

Hmm, having done this I realise that what I really need is a way to *find* things which are not in the dictionary, not all hyphenated words.

Try to replace the sub-function replace_word() that is inside the code of the example by this one, that does a replace when the compound word is NOT is the dict:

    def replace_word(wmatch):
        # if word1-word2 is not is not recognized by the dictionary, replace dash by em-dash
        with_em_dash = wmatch.group(1) + "—" + wmatch.group(2)
        if not dictionaries.recognized(wmatch.group()):
            return with_em_dash
        return wmatch.group()

[Phssthpok]

Originally Posted by lomkiri

Try to replace the sub-function replace_word() that is inside the code of the example by this one, that does a replace when the compound word is NOT is the dict

No, I did this -- the problem with this code is that I find every occurrence of a hyphenated word and then whether to replace it or not.

What I really want is a way to FIND the ones not in the dictionary and THEN decide whether to replace them. Out of 500 hyphenated words, maybe only 50 will need to be looked at as candidates for replacement, instead of looking at all 500. But I can't see any way to do that, so I'll just have to look at all 500.

[lomkiri]

Originally Posted by Phssthpok

What I really want is a way to FIND the ones not in the dictionary and THEN decide whether to replace them.

You could modify the regex-function so it selects the candidates and writes them down into a file (e.g.. not_in_dict.txt) without changing anything in your text.

Then you open this file in a text-editor and delete from it all but the occurrences you want to correct

Then you modify the regex-function to load this new file in a list, and to check each occurrence against this list : if it's present in the list, correct it in the text

Note: It could be more convenient to delete from the file only the occurrences you want to correct (deleting only more or less 50 instead of 450), in that case you'll have to do the contrary: correct only the ones which are NOT in the list.

[Phssthpok]

Originally Posted by lomkiri

You could modify the regex-function so it selects the candidates and writes them down into a file (e.g.. not_in_dict.txt) without changing anything in your text.

Then you open this file in a text-editor and delete from it all but the occurrences you want to correct

Then you modify the regex-function to load this new file in a list, and to check each occurrence against this list : if it's present in the list, correct it in the text

Note: It could be more convenient to delete from the file only the occurrences you want to correct (deleting only more or less 50 instead of 450), in that case you'll have to do the contrary: correct only the ones which are NOT in the list.

Ah, that's a good idea! I'll give it a whirl.

Of course, for the book that made me think about this issue, I have already gone through the 500 instances by hand and corrected the 50 that were wrong... but I'm sure it'll happen to me again!