How to fix missing spaces between dictionary words

[DVdm]

I have this book where many thousands of spaces are missing between words. For instance: hehad, thereis, ina, withme, hecame, theworld, ...
In the standard editor (or in some plug-in perhaps), is there a way to use a regex function that can (1) find words that are not present in the standard dictionary, but that consist of two words that are present in the dic, and (2) propose a change?
I have read the doc on Function mode for Search & replace in the Editor, but I don't immediately see how this could be done. I'm sure someone must have had this problem before...
Tia for any tips.

[theducks]

The editor finds misspelled words and proposes changes. (tick show only....)

One of those are to split into 2 words (not 100%, but 2 valid words)
Note: This does not find <tagged> runtogethers

[DVdm]

Originally Posted by theducks

The editor finds misspelled words and proposes changes. (tick show only....)

One of those are to split into 2 words (not 100%, but 2 valid words)
Note: This does not find <tagged> runtogethers

Yes, thanks, I had found that, but even this will takes many hours. I would like the editor to go ahead and do all corrections, provided there are indeed two words. For instance, for bethe its choices are Bethe, bathe, Lethe, be the, be-the,... , where only the 4th item is the one I need, two separate existing words.
I'd like to make all the changes in one blow, and then look for any induced mistakes.

[DVdm]

Thanks.
I replied, but my message doesn't turn up.

[Karellen]

I have found this issue in quite a few books. Sometimes it looks like a previous editor has simply deleted all the hyphens in the book.

I use the "Check Spelling" function and simply scroll down the list of all mis-spelled words and fix them.
Alt-F7 or Tools►Check spelling to access it.
Its a lot easier than trying to check every page of the book.

[phossler]

There's a RegEx function called 'SplitWords' that tries to divide using the dictionary

https://www.mobileread.com/forums/sh...ht=split+words
Post #9 by the master

I've never used it so let me know how it goes

import regex
from calibre import replace_entities, prepare_string_for_xml

def replace(match, number, file_name, metadata, dictionaries, data, functions, *args, **kwargs):
    def fix_word(m):
        word = m.group()
        if dictionaries.recognized(word):
            return word
        for i in xrange(1, len(word) - 1):
            a, b = word[:i], word[i:]
            if dictionaries.recognized(a) and dictionaries.recognized(b):
                return a + ' ' + b
        return word
    text = replace_entities(match.group(1))
    text = regex.sub(r'\b\w+\b', fix_word, text, flags=regex.VERSION1)
    text = prepare_string_for_xml(text)
    return '>' + text + '<'

[DVdm]

Originally Posted by phossler

There's a RegEx function called 'SplitWords' that tries to divide using the dictionary

https://www.mobileread.com/forums/sh...ht=split+words
Post #9 by the master

I've never used it so let me know how it goes

import regex
from calibre import replace_entities, prepare_string_for_xml

def replace(match, number, file_name, metadata, dictionaries, data, functions, *args, **kwargs):
    def fix_word(m):
        word = m.group()
        if dictionaries.recognized(word):
            return word
        for i in xrange(1, len(word) - 1):
            a, b = word[:i], word[i:]
            if dictionaries.recognized(a) and dictionaries.recognized(b):
                return a + ' ' + b
        return word
    text = replace_entities(match.group(1))
    text = regex.sub(r'\b\w+\b', fix_word, text, flags=regex.VERSION1)
    text = prepare_string_for_xml(text)
    return '>' + text + '<'

Of course I have to replace

return '>' + text + '<'

with

return text

but, using the search string

\b(\w+)\b(?![^<>{}]*[>}])

it only works with words that are in the dictionary - i.e. they are left unchanged. But when I encounter a word that is not in the dic, I get an error

NameError: name 'xrange' is not defined

[kovidgoyal]

change xrange to range in that function

[DVdm]

Originally Posted by kovidgoyal

change xrange to range in that function

Yes!

[DVdm]

Originally Posted by DVdm

Yes!

Using the regex function, it took me a few hours to fix the book, and then I realised something.
This was a book that I had found as a pdf, which I had converted to epub with Calibre. In an edit session I noticed that there were a bunch of paragraphs with some kind of hardcoded linefeeds. With a general search and replace, I replaced all linefeeds with nothing, effectively deleting them, and then did a global beautifying files.
Stupid. I should have replaced the linefeeds with spaces.
So I retrieved the original pfd from my backups, converted to epub, replaced all linefeeds with a space, and beautified all files. Ready.
Silly me!