KOReader: a document reader for PDF, DJVU, EPUB, FB2, HTML, ... (GPLv3)

[sadowski]

Thanks for responding. By inflection, do you mean a set of rules like "ies" --> "y" which in an English dictionary would allow to find "dictionary" when searching for "dictionaries"? Or do you mean brute force enumeration of all forms of a word within the dictionary, so that in this example it would contain both dictionary and dictionaries?

Pocketbook dictionaries allow this kind of substitution rules as part of the dictionary file (called "morphemes").

[Doitsu]

Originally Posted by sadowski

Thanks for responding. By inflection, do you mean a set of rules like "ies" --> "y" which in an English dictionary would allow to find "dictionary" when searching for "dictionaries"?

AFAIK, StarDict doesn't support rule-based inflection lookups. However, dictionary authors can define inflections/synonyms in StarDict dictionary source files with pipe symbols ('|'). For example:

go|goes|going|went
to move from one place to another

bring|brings|bringing|brought
to take or go with (someone or something) to a place.

[CaptainD]

Hi!

Thank you for the app. I have a few questions:

There is no html books in default Kindle book list. To open html, I have to launch
KUAL -> Koreader -> Start file manager -> .html file
Is it possible to view html in Kindle book list?

Some files could be opened in default reader and in Koreader (pdf, for example). For these books, can I choose app for reading?

Is it possible to highlight the text and make any clippings or comments? As I see, the only way to make notes in Koreader is bookmark.

[CaptainD]

Originally Posted by CaptainD

Hi!
Is it possible to highlight the text and make any clippings or comments? As I see, the only way to make notes in Koreader is bookmark.

I found it.
But there is a strange thing: highlight menu appears when I select few words from the book and it doesn't appears when I select single word.
The menu has only two active items: "Highlight" and "Translate".

[Markismus]

Originally Posted by sadowski

The "fuzzy" search algorithm that stardict implements can give inadequate results sometimes....

There seem to be 2 problems with this look-up algorithm:

1. If there is no exact match, stardict falls back to a fuzzy search,...

2. In most Europen languages, words roots can be found by manipulating endings, e.g., typically --> typical. This is language specific but makes dictionaries much more efficient.

Anyone else stumbled over this? Any suggestions?

On the koreader/sdcv page Chrox refers for info to a man page. In the description it states:

sdcv is simple, cross-platform text-base utility for work with dictionaries in StarDict's format. The word from "list of words" may be string with leading '/' for using Fuzzy search algorithm, with leading '|' for using full-text search, string may contain '?' and '*' for using regexp search....

Sdcv is called in line 67 in readerdictionary.lua:

 local std_out = io.popen(
    "./sdcv --utf8-input --utf8-output -nj "
    ..
    ("%q"):format(word)
 , "r")

From the man page the only switch I don't understand is "-nj". The "n" would be for non-interactive, but the "j" eludes me.

More importantly there is no leading "/" for Fuzzy search algorithm, nor leading "|" for using full-text search. So sdcv doesn't run in fuzzy logic mode for koreader.

This does however give you some room for experimenting with sdcv. You could simply alter line 67 and add at the end of the first part of the string a "|":

 "./sdcv --utf8-input --utf8-output -nj |"

I would be interested whether those changes have an impact on your problem.

As sdcv is apparently chosen because of simplicity a more complex approach to dictionary lookup would either mean that you'll have to write code to preformat the "word" or to implement an alternative command line utility with more features. The emphasis on command line use is that it can be called from within the lua-code with a one-line command! Anything more complex would require extra code to wrap around the utility and thus more time for execution.

[sadowski]

From the man page the only switch I don't understand is "-nj". The "n" would be for non-interactive, but the "j" eludes me.

More importantly there is no leading "/" for Fuzzy search algorithm, nor leading "|" for using full-text search. So sdcv doesn't run in fuzzy logic mode for koreader.

The "-j" has been added to this sdcv version and means "JASON output" to return the output somehow formatted for display.

Even if the "/" is not preceding the query, fuzzy search is the fall-back if no exact match is found, I looked this up in the source. The leading "|" would not help since it seems to mean full search in all text, not just the word list.

For many (European) languages it would be better to fall back to return a list of words that precede and follow the search query in lexicographical order. Even better would be if we would have language-specific rules how to modify the query to find word roots or singular for plural etc. Just the fuzzy search seems to be inadequate.

[hawhill]

It's "JSON", which is a data syntax.

Yes, sdcv does a fallback. And to be more exact, it is a levenshtein distance based search. That will always have it drawbacks. But then, it's what sdcv offers. sdcv is upstream code and isn't likely to implement more features soon (I guess).

I started an implementation of a stardict dictionary parser in Lua, but it's in no way finished and it will still need considerable development to bring about something useful with it.

@CaptainD: Can you file an issue for this in https://github.com/koreader/kpvbooklet ?

[Markismus]

Originally Posted by hawhill

I started an implementation of a stardict dictionary parser in Lua, but it's in no way finished and it will still need considerable development to bring about something useful with it.

Nice. I would love to take a look at the code and maybe help development along for a bit.

[sadowski]

Originally Posted by hawhill

I started an implementation of a stardict dictionary parser in Lua, but it's in no way finished and it will still need considerable development to bring about something useful with it.

Could some of this be easier done in the startdict source? I see in the sdcv code that some language-specific query modification is done for English, e.g., -ies/-y in lib/lib.cpp, LookupSimilarWord(). I just did not figure out how to cross-compile for Kobo, otherwise I would give it a try.

[Markismus]

Originally Posted by sadowski

Could some of this be easier done in the startdict source?...I just did not figure out how to cross-compile for Kobo, otherwise I would give it a try.

Developers can install not only the koreader files, but also the koreader-base files. This would allow you to build and emulate koreader, including sdcv.
Normally the koreader github has a reference to the current koreader-base (first line), but if you install them yourself you can change and compile to your liking. In the koreader-base files there is a similar link to the current sdcv ( sdcv-0.4.2 @ 9ee1c9e). So you could also install those files and start tweaking.

All the compiling and emulation would be taken care of by the makefiles already in place. And if your tweaks/coding would be as effective as you hope for, you could send a pull-request to koreader-sdcv and have discuss your changes and add them to koreader!

[PieMan597]

Hey Hawhill, nice reading app for my Kindle! Thanks!

[hawhill]

Thanks are appreciated. However, Koreader didn't get what it is by just one author. In fact, the majority of the code is the work of others - and was created in cooperation. Koreader is Open Source, and it is also created in an effort open to anyone. So let's say "thank you" to the whole development community!

[hawhill]

Originally Posted by sadowski

Could some of this be easier done in the startdict source? I see in the sdcv code that some language-specific query modification is done for English, e.g., -ies/-y in lib/lib.cpp, LookupSimilarWord(). I just did not figure out how to cross-compile for Kobo, otherwise I would give it a try.

Of course you can hack your way through the sdcv code. But don't dump the changes at koreader - better have them go upstream. Chances are, however, that people are not willing to staple more hacks on those that are already present. At least, I don't consider these hardcoded modifications a "clean approach". Without investigation, I simply guess that they were a fast way to scratch an itch.

Reimplementing sdcv in Lua serves another approach, however. At least that is the reason I started to do so. On Android builds of Koreader, it's a bit messy to start external processes. And it would simply be nice to have an implementation that is easy to hack on. E.g. a language-based plugin system could be so much easier to do in Lua, since Lua allows for a bit more dynamic approach to developing stuff. (not meant to start a language war. C++ is also a fine language.)

[Markismus]

Originally Posted by hawhill

But don't dump the changes at koreader - better have them go upstream.

I thought the Stardict development was quite dead. Is it still ungoing then?

[peaceridge]

I don't have enough bandwidth to go through all 47 pages, so please allow me this question. I want to try KoReader on my android Nexus 7 2013, but can't find it in the google store. How do I go about getting a copy? Thanks.