Question about dictionaries

[mzel]

Is the support of morphology (genders, conjugations, contractions, etc.) the function of a dictionary or the koreader application?

It is not so much of a problem for English, but it is for many other languages

[mergen3107]

AFAIK, Stardict (which engine is used in KOReader) does not support morphology. However it has fuzzy search, which looks for the most similar looking word if no exact match found. Sometimes it works, sometimes doesn’t. In the latter case I just tap and hold the word title in the dict popup in KOReade and type my required word manually.

[Doitsu]

Originally Posted by mergen3107

AFAIK, Stardict (which engine is used in KOReader) does not support morphology.

StarDict does support inflections. The KOReader StarDict engine does not.

EDIT: I was wrong, KOReader has been supporting .syn files since late 2019.

[mzel]

I guess you are talking about the .syn (synonyms) file. It is not really inflection. It requires you to list all the possible forms of the word as opposed to the list of rules of the language.
That means that if there is ~100 forms of the verb in Italian you need to provide 100 forms for each of the verbs as opposed to 100 rules for all the correct verbs combined

[Doitsu]

Originally Posted by mzel

I guess you are talking about the .syn (synonyms) file.

I was indeed referring to .syn files, which the KOReader StarDict engine doesn't support.

Originally Posted by mzel

That means that if there is ~100 forms of the verb in Italian you need to provide 100 forms for each of the verbs as opposed to 100 rules for all the correct verbs combined

AFAIK, there are no cross-platform Open Source dictionary engines that support defining POS-based morphology rules.
Having to define all forms for each entry may seem like a rather primitive method, but it works surprisingly well.

BTW, If you want to add inflections to your own StarDict dictionary, you might find Tvangeste's inflection word lists for English, French, Italian, German, Spanish, Portuguese, Polish and Russian helpful.

[NiLuJe]

Doesn't it? I recall a host of issues about sdcv being *slow* when dealing with synonyms, but handling them nonetheless .

[Galunid]

Yup, it should support it, at least according to the issue @NiLuJe mentioned: https://github.com/koreader/koreader/issues/5437

[Doitsu]

Originally Posted by NiLuJe

Doesn't it? I recall a host of issues about sdcv being *slow* when dealing with synonyms, but handling them nonetheless .

You are of course right. Apparently, KOReader has been supporting .syn files since 2019.

(I updated my initial post.)

[mzel]

Reading up on this forum and 3-4 others I came to the conclusion that my options are:
1) Generating .syn file out of the grammar rules from the link above or the .aff file from a Goldendict dictionary
2) Trying to build a command line Goldendict for Kobo and write a plugin for it in koreader
3) Implementing those same rules in .lua wrapper around sdcv and trying to find a closest match from koreader
4) finding a ready-made .syn file for the language - Italian in this case
5) Something else? Kindle was able to do a better job in this department. I mean the native Kindle reader with dictionaries built for it. The Italian-English dictionary was pretty good in this regard. The Italian-Russian was not perfect, but still better than what we have now under koreader. It uses the same initial vocabulary but handles inflections way better. I never tried to install the dictionaries under koreader on kindle
6) Forego all of the above and use manual entry plus a guesswork to arrive at the correct headword

All suggestions and comments are welcome

[Doitsu]

Originally Posted by mzel

4) finding a ready-made .syn file for the language - Italian in this case

AFAIK, .syn files are dictionary-specific index files that can't be re-used. They're automatically generated by StarDict Editor when you compile a Babylon GLS source file.

Originally Posted by mzel

5) Something else? Kindle was able to do a better job in this department.

You could unpack one of the free bilingual Oxford dictionaries that Amazon offers as optional downloads for eInk Kindle users with KindleUnpack and extract the inflection data.

Here's an example entry from the Italian-English Oxford dictionary:

<idx:orth value="abbacchiato">
    <idx:infl>
        <idx:iform name="" value="abbacchiata"/>
        <idx:iform name="" value="abbacchiate"/>
        <idx:iform name="" value="abbacchiati"/>
    </idx:infl>
</idx:orth>

The Babylon GLS equivalent is:

abbacchiato|abbacchiata|abbacchiate|abbacchiati

[mzel]

re 4) IMHO the only dictionary specific part of the .syn is the set of basic words. Otherwise it should be language specific. As far as I understand .syn is part of the input, not output for the Stardict converter.
re 5) That again would only be the intermediate point to create a .syn for Stardict. BTW do you have the link to those dictionaries?

[Doitsu]

Originally Posted by mzel

As far as I understand .syn is part of the input, not output for the Stardict converter.

No, the opposite is true. StarDict Editor will automatically generate a .syn file if the Babylon GLS source file contains inflections.
This also means that the .syn files are not interchangable.

You might want to compile a simple Babylon GLS source file yourself with StarDict Editor. (I attached a small test file to this post; the input file is fr_en_sample.babylon)

Originally Posted by mzel

BTW do you have the link to those dictionaries?

No, but if you own a registered eInk Kindle, you can download them for free.

[pazos]

Originally Posted by mzel

2) Trying to build a command line Goldendict for Kobo and write a plugin for it in koreader
3) Implementing those same rules in .lua wrapper around sdcv and trying to find a closest match from koreader

Contributions are always welcome.

Keep in mind a few things:

- Goldendict is not a dict format. Is an app supporting plenty of dict formats. You'll need to choose which one of the supported goldendict formats you want to support, peek into goldendict code and get something that works for that specific format.

- There's no need to build a commandline tool. A lua C module would be fine too.

- Lua is too slow for a dict app. The code needs to be written in C/C++. The interface between your code and KOReader can be coded as you want.

From the supported colordict formats slob and zim look the most interesting to me, but I don't know if they have the features you need.

The end goal is to have a program/library that works in all KO devices, which are mostly linux arm devices. Support for 3rd party apps is already available on devices that are intended to run apps (android, linux, mac)

[mzel]

Originally Posted by Doitsu

No, the opposite is true. StarDict Editor will automatically generate a .syn file if the Babylon GLS source file contains inflections.
This also means that the .syn files are not interchangable.

You might want to compile a simple Babylon GLS source file yourself with StarDict Editor. (I attached a small test file to this post; the input file is fr_en_sample.babylon)


No, but if you own a registered eInk Kindle, you can download them for free.

I did try, in more than 1 way and failed. I did download that dictionary, but kindleUnpack fails there with:

line 770, in process_all_mobi_headers
    raise unpackException('Book is encrypted')
lib.kindleunpack.unpackException: Book is encrypted



Error: Unpacking Failed

And DRM removal for that file does not work too. I tried with the Calibre plugin, and with the standalone Kindle DRM removal tool too. Both fail at some point. After that I found:
https://github.com/apprenticeharper/...ols/issues/276
so at least I am not alone

[kandwo]

I would recommend not relying on Koreader's internal dictionary, bur rather using GoldenDict, which works fine with inflections. You have to install both the dictionary (stardict or other compatible format) and a hunspell dictionary for the language so that it recognizes the inflected forms.

This works well for the languages that I've tried so far: Russian, Ukrainian and English.