[Editor Plugin] Traditional<->Simplified Chinese Convertor

[nnyljh]

And to add to my earlier post just now.... Some additional testing shows:
- Converting Simplified horizontal azw3 to Traditional vertical epub worked fine. However converting the resultant epub file to mobi, it loses the vertical and goes back to horizontal layout.

[Hopkins]

Sorry, I haven't been paying much attention to this thread lately.

I don't know much about the mobi format. If the book displays properly in a reading tool that handles vertical text such as Adobe Digital Editions, I would try using the KindleGen application from Amazon directly. Drag the epub into the application and check if there are any warning messages created as it produces the mobi file.

[DNSB]

Originally Posted by nnyljh

And to add to my earlier post just now.... Some additional testing shows:
- Converting Simplified horizontal azw3 to Traditional vertical epub worked fine. However converting the resultant epub file to mobi, it loses the vertical and goes back to horizontal layout.

Did someone add support for vertical text in mobi when I wasn't looking?

[Hopkins]

Edit - Removed comment. See correct answer below.

[jhowell]

Originally Posted by Hopkins

Converted to mobi format via Kindle Previewer 3
...
It displays vertically on the Kindle.

The Kindle Previewer uses kindlegen to create a Master MOBI file. It contains the old MOBI-7 format along with the newer KF8 (azw3) format in the same file. (Notice the relatively large file size.) KF8 supports vertical text and MOBI-7 does not.

Your device is using the KF8 format from the file. An older Kindle will not do that.

[Hopkins]

Ok, got it.

nnyljh should use Calibre to convert to AZW3 format rather than MOBI format in order to maintain the vertical text on a newer Kindle. I confirmed that MOBI format reverts back to horizontal display in Kindle Previewer.

[maviw40218]

I've been using this plugin for several months, and it works great. But there are two issues I'd like to raise based on my experience:

1. for "simplified to traditional" conversion, some characters were better unconverted as the mapping was 1-to-multiple (e.g. 里 could mean miles or inside sth.) - personally I feel like reading "衣服里" (unconverted) would be more acceptable than something like "100裏" (converted).
2. regional nuances were said to be addressed (I suppose this refers to the "use output target phrases if possible" option), but they don't quite work for me. I suggest implementing a custom mapping for fine-tuning (e.g. 澳大利亚 to 澳洲), w.r.t. a list maintained by the users themselves.

[Hopkins]

1. I don't think there is any hope of fixing the first problem using this software tool. In order to get the best conversion, a natural language program is needed to determine the parts of speech and the best character to use. There are GNU NLP tools out there, so it is certainly possible for someone to write such a thing.

2. I guess I could put in an option so the user could supply a list of conversions that take precedence over all others. I'll look into that.

There are a few things I need to do anyway (convert to python 3, create a Sigil version, update to the latest lists of words, etc.)

[Aliocantinea]

Hopkins, I love you plugin and just wanted to make sure it was safe to update calibre to 5.0. I see you mentioned converting to python 3 in your previous post, does the 2.4.0 Sept 25th 2020 include this conversion or should I hold out for the next plugin update?

[Hopkins]

I've updated and tested it on Calibre 5.01. It appears to work; it should be safe to update to the latest Calibre. The latest Calibre fully supports reading of vertical text.

Note: I didn't do much testing of the command line operation; so no guarantees there.

[Hopkins]

Updated version available. As always, the previous versions are available on Github.

Changes:

• HTML parser completely rewritten
• Updated conversion dictionaries to OpenCC versions as of 29 December 2022
• Added conversion of Mainland traditional hanzi into "simplified" Japanese Kanji
• Added more control over punctuation conversion when changing reading orientation
• Remove Kindle/Readium setting; this only affected punctuation choices and is no longer needed
• Added ability to change a portion of the current xhtml file by surrounding the desired text with HTML comments

  <!--PI_SELTEXT_START-->

  and

  <!--PI_SELTEXT_END-->

  If the end tag is omitted, processing goes to end of file

[Hopkins]

Bug fix update for conversion from mainland simplified to Taiwan traditional.

[espn5008]

Hi just to report a few wrongly converted Chinese characters while trying to convert from Simplified Chinese (Mainland China) to Traditional Chinese (Taiwan).

Original Wrong Correct
发射 髮射 發射
发现 髮現 發現
发生 髮生 發生
发送 髮送 發送

Thanks

[Hopkins]

The conversion from Simplified Chinese to Traditional Chinese has a fairly high failure rate. The algorithm scans the sentence from beginning to end looking for the largest string in its conversion dictionary. Proofreading the conversion results of Simplified Chinese to Traditional Chinese is mandatory.

For example:

The isolated word 发现 converts to 發現 correctly.

But a sentence containing a fragment 工头发现 will get converted to 工頭髮現 because the word 头发 is seen first and is converted into 頭髮 before the word 发现.

Were there any characters immediately before or after 发现 in your test case?

Ultimately, it may be better to use a different tool for Simplified Chinese to Traditional Chinese conversions. Conversion from Traditional Chinese to Simplified Chinese has a much much lower error rate.

[Hopkins]

Bug fix update for processing character references.