[Editor Plugin] Traditional<->Simplified Chinese Convertor

[Hopkins]

Currently, the Chinese language is written with two different standardized character sets. The Chinese mainland and Singapore officially use the simplified set while other areas (such as Taiwan and Hong Kong) continue to largely use the traditional set. This plugin will allow users to convert EPUB and AZW3 files between both formats.

If only text format changes are desired (such as flow direction or quotation mark types), character set changes can be omitted. This allows changes to non-Chinese texts such as Japanese.

Main Features

• Convert eBooks written in traditional characters into simplified characters
• Convert eBooks written in simplified characters into traditional characters
• Convert regional words and idioms used in the source material to those words and idioms used in the destination material
• Convert individual sections or the entire book
• Update metadata and table of contents
• Convert text direction to vertical or horizontal
• Provides command line processing for batch operations
• This is an editor plugin so users can make changes in case the conversion is not perfect. Conversions from simplified to traditional should always be proofread.

Testing Platforms

• Windows 10 (64 bit) - Calibre version 3.34
• Linux Kubuntu 16.04 (64 bit) - Calibre 2.71
• Amazon Kindle Previewer

Note:

• This plugin incorporates python code from opencc-python (https://github.com/yichen0831/opencc-python). That code was licensed under Apache License 2.0 and is thus compatible in Calibre with GPL 3
• The plugin also uses python code from DiapDealer's Editing Toolbag (https://www.mobileread.com/forums/sh...d.php?t=251365)
• The text difference display is not particularly useful since entire lines are highlighted when only one character has changed

Github repository link

Command Line Interface(CLI)
Details:

Spoiler:

Example: overwrite all ebook files in a directory from Mainland simplified into Taiwan traditional (also change to East Asian quote marks and vertical text orientation) add in a "V" suffix to the file name and optimizing for display on the Chrome Readium reader:

calibre-debug --run-plugin "Chinese Text Conversion" -- -ol tw -il cn -d s2t -od out -qt e -td v -tdo r -a V *.epub *.azw3

Example: overwrite all epub files in a directory from Taiwan traditional into Mainland simplified, but don't actually perform the write. Just print what would happen:

calibre-debug --run-plugin "Chinese Text Conversion" -- -ol cn -il tw -d t2s -t my_chinese_epub_dir/*.epub

usage: calibre-debug --run-plugin "Chinese Text Conversion" -- [-h] [-il {cn,hk,tw}] [-ol {cn,hk,tw}]
[-d {t2s,s2t,t2t,none}] [-p] [-qt {w,e,no_change}]
[-sq] [-td {h,v,no_change}] [-tdo {r,k,none}] [-v]
[-t] [-q] [-od OUTDIR_OPT] [-a APPEND_SUFFIX_OPT]
[-f] [-s]
ebook-filepath [ebook-filepath ...]

Convert Chinese characters between traditional/simplified types and/or change
text style. Plugin Version: 2.3.0

positional arguments:
  ebook-filepath         One or more ebook filepaths - UNIX style wildcards
                        accepted

optional arguments:
  -h, --help            show this help message and exit
  -il {cn,hk,tw}, --input-locale {cn,hk,tw}
                        Set to the epub origin locale if known (Default: cn)
  -ol {cn,hk,tw}, --output-locale {cn,hk,tw}
                        Set to the epub target locale (Default: cn)
  -d {t2s,s2t,t2t,none}, --direction {t2s,s2t,t2t,none}
                        Set to the epub conversion direction (Default: none)
  -p, --phrase_convert  Convert phrases to target locale versions (Default:
                        False)
  -qt {w,e,no_change}, --quotation-type {w,e,no_change}
                        Set to the epub origin locale if known (Default:
                        no_change)
  -sq, --smart_quotes   Use smart quotes if applicable (Default: False)
  -td {h,v,no_change}, --text-direction {h,v,no_change}
                        Set to the epub origin locale if known (Default:
                        no_change)
  -tdo {r,k,none}, --text-device-optimize {r,k,none}
                        Optimize text for device (Default: none)
  -v, --verbose         Print out details as the conversion progresses
                        (Default: False)
  -t, --test            Run conversion operations without saving results
                        (Default: False)
  -q, --quiet           Do not print anything, ignore warnings - this option
                        overides the -s option (Default: False)
  -od OUTDIR_OPT, --output-dir OUTDIR_OPT
                        Set to the epub output file directory (Default:
                        overwrite existing epub file)
  -a APPEND_SUFFIX_OPT, --append_suffix APPEND_SUFFIX_OPT
                        Append a suffix to the output file basename (Default:
                        )
  -f, --force           Force processing by ignoring warnings (e.g. allow
                        overwriting files with no prompt)
  -s, --show            Show the settings based on user cmdline options and
                        exit (Default: False)

Installation Steps:

Download the attached zip file and install the plugin/add to context menu or toolbar/restart Calibre as described in the Introduction to plugins .

Operation:

From the main Calibre window, select a book and then press the "Edit book" icon on the toolbar. The editor will open. Press the "plugins" text on the editor toolbar and select the plugin.

Special Notes:

• Requires calibre v2.55 or higher
• No testing has been done on OS X systems
• Keep a copy of the original file. Round trip conversions (i.e. traditional->simplified->traditional) will probably not recover the original version. Also, since characters are being replaced, it's possible the font in your eBook reader may not have all the necessary glyphs
• Metadata changes made via the GUI do not update the main Calibre database. They will be overwritten once the editor is re-opened. Consider using the 'Save a copy' option
• The Calibre book viewer does not properly support vertical text display. Use Chrome Readium or Kindle instead.

Version History:

Spoiler:

• Version 1.0.0 - 24 Jun 2016. Initial release
• Version 1.1.0 - 27 Jun 2016. Improved speed
• Version 1.2.0 - 29 Jun 2016. Correct conversion, turn on compression for the plugin zip file
• Version 2.0.0 - 10 Nov 2016. Added command line processing, now also update TOC and metadata, updated conversion dictionaries
• Version 2.0.1 - 24 Jan 2016. Updated conversion dictionaries to latest at OpenCC project. Modified using chihchun's changes to allow plugin to work with more Calibre versions. Corrected minimum version.
• Version 2.1.0 - 19 Feb 2017. Added option to also convert quotation mark style to match target. Not yet added to the command line version.
• Version 2.1.1 - 5 Aug 2017. Correct exception that occurred when processing an entire book. See Github issue #3 for details.
• Version 2.2.1 - 22 Aug 2017 - Added vertical text orientation and epub quotation mark optimization for Readium and Kindle viewers. Kindle Previewer 3 must be used to convert epub files into Kindle mobi files.
• Version 2.2.2 - 30 Aug 2017 - Corrected CSS file for EPUB->AZW3 conversion. See Github issue #5 for details.
• Version 2.2.3 - 31 Aug 2017 - Improved speed for vertical text conversion optimization. See Github issue #5 for details.
• Version 2.2.4 - 15 Sep 2017 - Fix some CSS issues with vertical text. See Github issue #5 for details.
• Version 2.3.0 - 4 Oct 2017 - Add full support for AZW3 files. See Gihub issue #6 for details
• Version 2.3.1 - 13 Nov 2017 - Allow the settings dialog to resize by adding scroll bars.
• Version 2.3.2 - 24 Nov 2018 - Improve conversion speed. Default to convert entire book
  

[kovidgoyal]

Thanks, I have added it to the plugin index.

[GameMonsters]

Wow. Thank you. I finally can check out simplified Chinese books from my library. I have been waiting for a useful tool like this. Thank you SO MUCH.

[howardtang]

This plugin is really useful, and I use it all the time. However, I have two little recommendations for you.

1) this plugin can't convert the Chinese words inside metadata and table of contents
2) it would be perfect if there is a bulk convert function

i look forward to the next update~

[Hopkins]

Those changes look like they would be very useful. I have spare time, so I think I can take a quick cut at it next week.

[Hopkins]

Plugin updated:

- Added command line processing to support batch processing
- Updated dictionary (txt) files based on OpenCC changes:

Change 1
Change 2

- Plugin now updates Table of Contents (TOC) and Content metadata. The GUI updates both only if the "Entire eBook" option selected. Command line always does the entire book.

[el_dheeb]

I have tried installing this plugin multiple times, both through Calibre and manually. Each time I get a message that the Plug in is installed and and to restart the machine. However Once i restart the Machine the plug in is listed as not installed. Here is the log after I install it.

calibre Debug log
calibre 2.76 embedded-python: True is64bit: False
Windows-8-6.2.9200 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '8', '6.2.9200')
Python 2.7.9
Windows: ('8', '6.2.9200', '', 'Multiprocessor Free')
Successfully initialized third party plugins: DeDRM (6, 0, 8) && FanFicFare (2, 7, 0) && Overdrive Link (2, 7, 0) && Chinese Text Conversion (2, 0, 0)
devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 100.861627907 x 101.07357513
Starting up...
FFF: INFO: 2017-01-10 22:21:23,918: calibre_plugins.fanficfare_plugin.prefs(201): Attempting to read settings from predecessor--FFDL
FFF: INFO: 2017-01-10 22:21:23,918: calibre_plugins.fanficfare_plugin.prefs(206): Using default settings
FFF: DEBUG: 2017-01-10 22:21:24,094: calibre_plugins.fanficfare_plugin.fff_plugin(207): Plugin FanFicFare macmenuhack file_path:C:\Users\el_dheeb\AppData\Roaming\calibr e\plugins\fanficfare_macmenuhack.txt
Started up in 57.20 seconds with 69 books
Downloading plugin zip attachment: https://code.calibre-ebook.com/plugins/275572.zip
Installing plugin: C:\Users\el_dheeb\AppData\Local\Temp\calibre_iunpo z\fawk5u.zip
Downloading plugin zip attachment: https://code.calibre-ebook.com/plugins/275572.zip
Installing plugin: C:\Users\el_dheeb\AppData\Local\Temp\calibre_iunpo z\irrznj.zip
Downloading plugin zip attachment: https://code.calibre-ebook.com/plugins/275572.zip
Installing plugin: C:\Users\el_dheeb\AppData\Local\Temp\calibre_iunpo z\4bjcyc.zip
Starting debug executable: C:\Program Files (x86)\Calibre2\calibre-debug.exe

[Hopkins]

I just noticed I have the same problem. When I look at the "User Plugins" dialog window in the main program, I see that the plug is in the "Not Installed" list. But when I open the editor, I see that the plug is available (and works) under the "Plugins" menu.

Try this:

1. Select a book in the main Calibre program
2. Open the editor by clicking on the "Edit book" icon
3. Click on the "Plugins" menu item in the editor

Check if there is an entry called "Convert Chinese Text Simplified/Traditional".

It's possible that the plugin is actually installed, but not properly registering with the main library application. Let me know what happens.

Edit:

Also, try this in the main Calibre window:

1. Click on the "Preferences" icon
2. Click on the "Plugins" icon which is at the bottom of the newly opened "Preferences" dialog
3. Click on the "Edit Book Tool plugins" pull-down arrow

The plugin shows up in the list in this dialog in my case.

[davidfor]

@Hopkins: I think this means you have a name mismatch somewhere in the plugin. From memory, when I did this with one if my plugins, I had to uninstall it before the correct name would take.

[Hopkins]

I think I see the issue. The name given in the plugin index and displayed via the "User Plugins" dialog is "Traditional<->Simplified Chinese Converter" where as the name I used in the plugin is "Chinese Text Conversion". I believe I changed the name at one point because the command shell processor and command line parser got annoyed when I used characters such as "<", "-", and ">".

I think the name in the plugin index may need to be changed.

For anyone writing plugins that may eventually use the command line interface:

• Avoid whitespace in the name
• Avoid using any character that might get interpreted by the shell or parser (e.g. <, >, - |, ! $, #, ?, /, \)

[el_dheeb]

Your right, it did install. Thank you much.

[chihchun]

Hi,

tested with calibre 2.55.0+dfsg-1 on Ubuntu 16.04, the editor could not load the plugin.
Not sure if I missed some dependency

(process:10492): Gtk-WARNING **: Locale not supported by C library.
Using the fallback 'C' locale.
File "/usr/bin/calibre-parallel", line 20, in <module>
sys.exit(main())
File "/usr/lib/calibre/calibre/utils/ipc/worker.py", line 190, in main
result = func(*args, **kwargs)
File "/usr/lib/calibre/calibre/gui_launch.py", line 76, in gui_ebook_edit
gui_main(path, notify)
File "/usr/lib/calibre/calibre/gui2/tweak_book/main.py", line 37, in gui_main
_run(['ebook-edit', path], notify=notify)
File "/usr/lib/calibre/calibre/gui2/tweak_book/main.py", line 71, in _run
main = Main(opts, notify=notify)
File "/usr/lib/calibre/calibre/gui2/tweak_book/ui.py", line 259, in __init__
self.create_actions()
File "/usr/lib/calibre/calibre/gui2/tweak_book/ui.py", line 495, in create_actions
create_plugin_actions(actions, toolbar_actions, self.plugin_menu_actions)
File "/usr/lib/calibre/calibre/gui2/tweak_book/plugin.py", line 166, in create_plugin_actions
for tool in load_plugin_tools(plugin):
File "/usr/lib/calibre/calibre/gui2/tweak_book/plugin.py", line 118, in load_plugin_tools
traceback.print_stack()

[chihchun]

It's API changed, the patch[1] works for me. :-)

[1] https://gist.github.com/chihchun/eee...9c1f62e8a84f39

[Hopkins]

I see what happened. It looks like the API was changed in version 2.59 back in June 2016. I added the TOC stuff in the last plugin update in November 2016 but never tested on early versions. I'll incorporate your change and update the minimum version information to 2.55 (though it may very well work for earlier versions)

I was looking for an excuse to update the changes to the conversion dictionaries that were made to OpenCC since my November update:

https://github.com/BYVoid/OpenCC/com...ata/dictionary

I'll also add some text to my original post explaining how users can update their conversion dictionaries (they are just UTF-8 text files) in case I'm not around in the future...

[chihchun]

Thank you, it would be great if you could host your code on github.com or somewhere that I can propose a pull request. :-)