kobopatch request thread

[AeroFengBlade]

Could I request a patch for enabling the "Japanese" dictionary lookup functionality for Chinese dictionaries? (basically enabling DictionaryParser::htmlForWordJapanese() for not only Japanese, but also Chinese)

I'm using a custom Chinese-English dictionary, but I have to masquerade it as a Japanese-English dictionary by renaming it dicthtml-ja-en.zip. (Advice from this thread, thanks tshering for all the pointers).

The issue is that I also want a Japanese-English dictionary. I know I can workaround by masquerading them as ja-es or ja-it dictionaries, but I think it's possible to make a patch to support both ja-en and zh-en dictionaries.

I read through the code and docs for dictutil. From my understanding:
- For English and other languages: uses 2-character prefixes, calls DictionaryParser::htmlForWordNonJapanese()
- For Japanese: uses 1-character prefixes DictionaryParser::htmlForWordJapanese(). Chinese should also use this since they both use kanji/hanzi; however, this code path is hardcoded for only the 'ja' locale.

I poked around in libnickel.so using objdump/nm (with some assistance from an LLM), and I think it all boils down to the hardcoded check in DictionaryProvider::getDefinition():

// Reconstructed code from disassembled thumb code
QString locale = dictionary.leftLanguage();
bool useJapanesePath = (locale == QLatin1String("ja"));
htmlForWord(word, useJapanesePath); // branches on htmlForWordJapanese() vs htmlForWordNonJapanese()

So far, that's about all I can verify by myself, but I think changing the code to also check for "zh" should basically do what I want.

I asked my LLM to take a look at my libnickel.so and draft a patch for kobopatch (Kobo Libra Colour, FW 4.45.23697). However there's a high potential it could be hallucinating since I don't have the proper ARM assembly knowledge to verify anything and I'm too scared to apply it.

Maybe someone here with more expertise could take a look at this to see if it holds water?

Spoiler:

# Technical details:
#   In DictionaryProvider::getDefinition(), the original code does:
#     QString locale = dictionary.leftLanguage();
#     bool useJapanesePath = (locale == QLatin1String("ja"));
#     htmlForWord(word, useJapanesePath);
#
#   The comparison is done by calling QString::operator==(QLatin1String) with a
#   hardcoded "ja" string. We replace the entire QLatin1String setup + comparison
#   call (20 bytes) with an inline check of the first UTF-16 character of the locale
#   string via Qt5's QString internal layout:
#
#     QArrayData* d = locale.d;
#     QChar firstChar = *((QChar*)((char*)d + d->offset));
#     bool useJapanesePath = (firstChar == 'j' || firstChar == 'z');
#
#   This matches "ja" (Japanese) and "zh" (Chinese) while not matching "en", "fr", etc.
#
# Assembled Thumb-2 replacement (20 bytes):
#   ldr  r0, [r5]         @ r0 = QString d-pointer
#   ldr  r3, [r0, #12]    @ r3 = d->offset (QArrayData layout)
#   add  r0, r3           @ r0 = pointer to UTF-16 char data
#   ldrh r0, [r0]         @ r0 = first UTF-16 character
#   cmp  r0, #0x6a        @ compare to 'j'
#   beq  +6               @ match -> skip to result (r0 = nonzero = true)
#   cmp  r0, #0x7a        @ compare to 'z'
#   beq  +2               @ match -> skip to result (r0 = nonzero = true)
#   movs r0, #0           @ no match -> r0 = 0 = false
#   nop                   @ padding
#   ; falls through to existing: mov r9, r0 (unchanged, saves result)

Enable Chinese dictionary fast lookup:
  - Enabled: no
  - Description: |
      Enables the fast 1-character prefix dictionary lookup for Chinese (zh) dictionaries.
      Without this patch, Chinese dictionaries using 1-char prefixes must be disguised as
      Japanese (dicthtml-ja-en.zip). With this patch, dicthtml-zh-en.zip works natively
      with fast lookup when the dictionary is built with 1-character prefixes.
  # Replace the QLatin1String("ja") setup + QString::operator== call with an inline
  # first-character check that matches both 'j' (Japanese) and 'z' (Chinese).
  - ReplaceBytes:
      Base: "DictionaryProvider::getDefinition(QString const&)"
      Offset: 28
      FindH: 2B 4B 02 22 BA 60 28 46 7B 44 FB 60 94 E8 06 00 83 F7 76 E3
      ReplaceH: 28 68 C3 68 18 44 00 88 6A 28 03 D0 7A 28 01 D0 00 20 00 BF

[geek1011]

You should be good to try it, the worst that'll happen in this case is a crash when you use the dictionary. The logic looks sane, but I didn't actually look at the instructions surrounding it, so I can't tell you if it'll work.

[AeroFengBlade]

Thanks for the reassurance, I just applied the patch on 4.45.23697 on my Libra Colour, works like a charm! I'm using the Chinese-English dictionaries I generated which I posted here. I kept the name as dicthtml-zh-en.zip and it works blazingly fast since it is using Japanese kanji lookup logic.

By the way, I was playing with your libnickel dictword-test tool, and I realized the function signatures and symbol names have changed since 2020. It looks Rakuten did some refactoring for the dictionary lookup logic, so the non-Japanese vs Japanese lookup is just a simple boolean flag passed to DictionaryParser::htmlForWord(QString const&, bool), so Japanese lookup is trivial to implement now. I submitted a PR here: https://github.com/pgaskin/kobo-mods/pull/8

[AeroFengBlade]

Never mind, I don't think this patch is necessary anymore. I came across information about the prefix_exceptions trie file in your mobileread thread and the dictutil github thread, and was able to build an index of Chinese word -> single-char hanzi prefixes (no variants needed).

This skips the Japanese lookup path completely, but still works and is pretty fast.

[niclake]

I have a very nitpicky patch request - the ability to show the <page> of <totalPages> AND percentage in the footer at the same time.

I know that I could turn on the progress indicator bar and get some of that, but it's just a preferred option I would love to see. I'm sure others would utilize it to a degree as well.

[bluepeter]

Originally Posted by niclake

I have a very nitpicky patch request - the ability to show the <page> of <totalPages> AND percentage in the footer at the same time.

I know that I could turn on the progress indicator bar and get some of that, but it's just a preferred option I would love to see. I'm sure others would utilize it to a degree as well.

Yes, that would be nice to have.

As it happens, I have both visible in my status bar when I use KOReader.

[fogice]

Originally Posted by niclake

I have a very nitpicky patch request - the ability to show the <page> of <totalPages> AND percentage in the footer at the same time.

I know that I could turn on the progress indicator bar and get some of that, but it's just a preferred option I would love to see. I'm sure others would utilize it to a degree as well.

Check out the (poorly named) Kobo Tweaks extension. I'm finding it to be a more stable and more flexible replacement for NickelClock on my Libra 2.

If you go to the GitHub page linked above, you will see that their example footer has the book title, page number, percent read, AND time remaining as the left footer, and battery and clock as the right footer. And you aren't forced to have the kitty in the folded bookmark image. I also like that it can hide the battery until it drops to a certain percentage (I have it at 20%).