Find Repeated/Duplicate Text

[Quoth]

Originally Posted by mikapanja

That works in case you know what you are looking for. But what if you don't, which was my original thought? E.g. if you want to know if any 5-word group is repeated in the text.

Tricky.
Proof read with your Deja Vu set to 11?

The concordance tool sounds good to detect repetitive writing, which really annoys people if it slips past the proof reading.

[Quoth]

Originally Posted by Doitsu

For those kind of searches you'll need to use a concordance tool. For example, Laurence Anthony's AntConc (freeware).

• Unzip the epub file.
• If the folder contains .xhtml files change their file extensions to .html.
• Open AntConc, select Open file(s) as 'Quick Corpus', select .html as the file type and then select the extracted .html files.
• Click the N-Gram tab, select the desired number of words and click Start.

I've attached a sample screenshot of the output.

Obviously, copy and paste all the chapters/files to one mega file* as the repeat in a novel (likely unwanted if you are the author or official editor) is probably in a separate file.

Indeed any search tool is useless unless you know the exact repeating text.


[* Unless it has a project / session mode that remembers previous files?]

[JSWolf]

Originally Posted by Quoth

Obviously, copy and paste all the chapters/files to one mega file* as the repeat in a novel (likely unwanted if you are the author or official editor) is probably in a separate file.

Indeed any search tool is useless unless you know the exact repeating text.


[* Unless it has a project / session mode that remembers previous files?]

It would be best to convert the ePub to text and then run it through AntConc.

[mikapanja]

Originally Posted by Doitsu

For those kind of searches you'll need to use a concordance tool. For example, Laurence Anthony's AntConc (freeware).

Tried it following your instructions - while setting the frequency to 2 - and it found an erroneus duplication of an unknown word group in no time. I knew it was somewhere in the text, but had no idea what it was and where.

And double-clicking a hit gives you short context. And double-clicking the context brings up the whole text, with repetition highlighted. Perfect.

Doitsu, thank you so much!

Originally Posted by Quoth

Indeed any search tool is useless unless you know the exact repeating text.

See above.