[GUI Plugin] TextDiff

[feuille]

[GUI Plugin] TextDiff - Version 1.3.1 - 09-27-2025

A Calibre GUI plugin for finding text differences in two book formats.

Main features:
--------------
This plugin shows the differences between two selected book formats.
The formats are first converted to text format (even if the source format is already text) with Calibre's convert utility (https://manual.calibre-ebook.com/gen...k-convert.html).
If the conversion fails, the format has no text content (as scanned PDF files) or Calibre cannot find an appropriate conversion tool (as Microsoft wordconv).
Then the text files obtained this way are read into memory and possibly edited (removing blank lines, soft hyphens, ...).
Then the compare is done with Python's DiffLib (https://docs.python.org/3/library/difflib.html).
The ratio gives a measure for the similarity of the two texts. 1.0 means the texts are identical, a value near 0.0 means that the texts are complete different.
The last thing may also occur, when the source format has no text content (as scanned PDF files). Then one should create a new book format (text) with an extra OCR process.

The detailed workflow is as follows:
1. Select a book with at least two formats or two books with at least one format each to compare.
2. Chose two formats.
3. Chose the output format and other comparison options.
4. Hit "Compare".
5. The formats are converted and compared and the result is displayed in the output window. A ratio is also computed and displayed.
6. If wished, copy the comparison output to the clipboard and/or save it to a file and/or save it as book with an suitable format (HTML or text).

If you want to compare other formats, repeat step 1 and hit the "Refresh formats" button. Then repeat steps 2 - 5.
The "Compare"-Dialog is modeless, what permits to move it around and touch the Calibre screen.

Limitations:
------------
- The converted formats are stored as strings in memory, so extreme large formats may run out of memory.

Version History:

Spoiler:

----------------
Version 1.3.1 - 09-27-2025
- Polish translation (thanks to bravosx)
- Some more character replacings: Three consecutive periods to three dot leader (HORIZONTAL ELLIPSIS).
Version 1.3.0 - 09-21-2025
- Spanish translation (thanks to dunhill)
- removing soft hyphens in input text
Version 1.2.6 - 10-27-2024
- Some more character replacings.
Version 1.2.5 - 04-23-2024
- Explanation for save diff result as book added.
Version 1.2.4 - 01-07-2024
- Fixing an typo in version 1.2.3 (causes an error, when selecting other output types than "HTML"; thanks to Zillion_).
Version 1.2.3 - 12-28-2023
- Check wether the pdf format is readable ((encrypted pdf's, pdf's with no text layer).
- Substitute different quotes and dashes characters with standard characters before diff (optional).
- Debug print optional.
Version 1.2.2 - 06-29-2023
- Disable buttons to save diff result until a result is generated (Thanks to Robert1a)
Version 1.2.1 - 03-23-2023
- Switch between context line processing by plugin or by Difflib
Version 1.2.0 - 03-22-2023
- Abort compare with message if convert has no result.
- Hide identical lines, but with the option to display a number of context lines. Closes enhancement request #1.)
Version 1.1.2 - 02-03-2023
- Adding double-quotes for the --sr1-search value: --sr1-search "(?m)^\s*$"
to avoid "syntax error near unexpected token \`('" on Mac. (Thanks to irinel-dan.)
Version 1.1.1 - 11-30-2022
- Handle save file dialog with no user path/file choice.
Version 1.1.0 - 11-26-2022
- Changed tool button behavior: show compare dialog when icon clicked, show menu when arrow clicked (thanks to Comfy.n)
- Inverting HTML/CSS back colors (highlighting diffs) in dark mode (thanks to Comfy.n and Kovidgoyal)
Version 1.0.0 11-20-2022
- Initial release.

Installation:
-------------
Download the attached zip file and install the plugin as described in the plugins thread on mobileread.
You need to add the calibre path to your $PATH variable.

To report Bugs and suggestions:
-------------------------------
If you find any issues or have suggestions, please report them on GitHub or in the MobileRead Forum.

[JSWolf]

Originally Posted by feuille

Planned Features:
-----------------
- Custom characters to ignore ("char junk", e. g. "" vs. »«)

I hope when this is implemented that there will be an option to turn it off. I do want to know if the quotes are different.

Limitations:
------------
- The converted formats are stored as strings in memory, so large formats may run out of memory.

This does need to be addressed as some computers are only 32-bit or may not have enough memory. IMHO, store both text files on disk and compare that way instead of fully loading both text files into memory and then comparing.

Overall, I do like the idea of this plugin. Thanks.

[ownedbycats]

Originally Posted by JSWolf

I hope when this is implemented that there will be an option to turn it off. I do want to know if the quotes are different.

"Custom" seems to imply it'll be user-defined?

But yes, the memory limitation seems... not great. I'd use temp files.

This'll be a useful plugin though

[JSWolf]

Originally Posted by ownedbycats

"Custom" seems to imply it'll be user-defined?

But yes, the memory limitation seems... not great. I'd use temp files.

This'll be a useful plugin though

My laptop does have 16GB, but most don't. My Surface Pro 2 is only 8GB.

I do think the memory usage should be fixed right away.

[chaley]

Originally Posted by ownedbycats

But yes, the memory limitation seems... not great. I'd use temp files.

Originally Posted by JSWolf

My laptop does have 16GB, but most don't. My Surface Pro 2 is only 8GB.

I do think the memory usage should be fixed right away.

The plugin is keeping TXT files in memory. According to this thread on Quora, 1 GB is approximately 900,000 pages, or 179 million "standard" words (5 single-byte characters plus a space). If the text is 100% UTF-8 extended characters then using the same assumptions as in the thread there are 11 bytes per word. A GB would be 98 million words or 488,000 pages.

I don't know how many people will be comparing 900,000 page books, or even 488,000 page books.

[feuille]

Originally Posted by JSWolf

I hope when this is implemented that there will be an option to turn it off. I do want to know if the quotes are different.

There will be a option to turn it on

[JSWolf]

Originally Posted by chaley

The plugin is keeping TXT files in memory. According to this thread on Quora, 1 GB is approximately 900,000 pages, or 179 million "standard" words (5 single-byte characters plus a space). If the text is 100% UTF-8 extended characters then using the same assumptions as in the thread there are 11 bytes per word. A GB would be 98 million words or 488,000 pages.

I don't know how many people will be comparing 900,000 page books, or even 488,000 page books.

Do you think this could be a problem for a computer running a 32-bit version of Calibre where you only get A 3GB chunk to work with?

[ownedbycats]

Originally Posted by chaley

The plugin is keeping TXT files in memory. According to this thread on Quora, 1 GB is approximately 900,000 pages, or 179 million "standard" words (5 single-byte characters plus a space). If the text is 100% UTF-8 extended characters then using the same assumptions as in the thread there are 11 bytes per word. A GB would be 98 million words or 488,000 pages.

I don't know how many people will be comparing 900,000 page books, or even 488,000 page books.

Just curious: how would this work for PDFs?

While I probably wouldn't be running it specifically on those books, PDFs downloaded from the Internet Archive use some sort of layering compression that means that pdftotext can extract gigabytes of image layers into the temp folder alongside the text layer. (This happens when indexing for FTS or running the word count plugin, which should only require the text layer.) Would it try to keep all that in memory?

[chaley]

Originally Posted by JSWolf

Do you think this could be a problem for a computer running a 32-bit version of Calibre where you only get A 3GB chunk to work with?

Possibly, but a vanishingly small one. A user would need to be running a 32 bit version (not possible in calibre 6) and be comparing 2 books in excess of a few hundred thousand pages.

If I were the developer then doing the diffs using files would be low priority, especially if the underlying tool didn't directly support it.

FWIW: calibre keeps the entire db in memory.

@owndbycats: it converts to txt, which by definition contains only text. Where it gets the text from a layered pdf is another issue.

[ownedbycats]

Yeah, just mentioned it because other things that I thought should be text-only (full-text indexing and the Count Pages plugin) I've seen it extracting the images into temp.

[JSWolf]

I've given TextDiff a try and it's pretty good. Thanks for creating this plugin.

[feuille]

Originally Posted by JSWolf

Thanks for creating this plugin.

You're welcome

[Comfy.n]

Thank you for this plugin! I plan to use it more, in the future, to compare Orwell's 1984 translations, but I just gave it a try for testing purposes and noticed the HTML output to file will be great for this use case, as it displays the text differences side-by-side in a very synchronized fashion.

some quick notes:

• The toolbar button click could default directly to the "Compare" action, instead of triggering the menu opening, IMO...
  
• Calibre UI will hang for a while during TextDiff, so at first I wasn't sure that this was just a temporary halt.
  
• The "Save output as ebook" action sometimes returns an error, I forgot to take note of what it was about
  
• Under Dark theme, it's best to use the "Save output to file", as the on-screen output isn't tuned to that color palette

[feuille]

Thank you Comfy.n for the hints! I'll try to fix that. For a better feedback of the program status I had already considered a progress bar.

[feuille]

Hello Comfy.n,

I implemented two of your hints (see history).

In order to calculate the display time of a progress bar, I stopped intermediate times in the program flow. To my surprise, the HTML rendering of the text browser widget consumes 2/3 of the total runtime! I may implement in a future release a workaround like stepwise asynchronous loading with a timer or so, but I need to do some research on that.