New dictionary format of firmware 2.14

[sento]

I'm trying to create my own dictionary. My problem is to generate a reference from one word to another. If i'm searching for a declinated form in the original dictionary, it is able to point to the definition of the base form. So i thought it must be possible to get such a reference working in my dictionary, but i don't get it working.
I also read the help "How to create Kobo Dictionaries", which says it's not possible to link from one definition to another one. What puzzles me. Because the original dictionary seems to do exact this, if i'm searching for a declinated form.

[AlPe]

In the lookup process there are two components: 1) the keywords actually present in the dictionary; 2) the lookup algorithm.

It might well be possible that the lookup algorithm implemented in the Kobo performs some language-dependent operation (like stemming), which seems to act like if there were "linked" keywords, to use your terminology.

Example 1: assume the lookup algorithm does a simple "search" for the exact work you select in the GUI: if you look for "friends" but your dictionary contains only "friend", then the result set will be empty. If the dictionary also contains the keyword "friends", pointing at the same definition of "friend", then it will retrieve it.

Example 2: now assume that the lookup algorithm DOES perform some more sophisticated operation, like stemming. You look for "friends", the stemmed version is "friend", hence the definition of "friend" will show up, even if you have searched for the plural and even if "friends" it is not present in the dictionary index.

Clearly stemming is a heuristic mechanism, which works well for some languages but it does not work for others, and it is just one of the possible approaches to the dictionary lookup problem.

(I have not had the chance of throughfully investigate the behaviour of my Kobo Glo to infer what it exactly does when you search for a term in the dictionary.)

[gouni]

@ Sento Kobo exemple definition in red

<a name="mazouter"/><variant name="mazouta"/><variant name="mazoutai"/><variant name="mazoutaient"/><variant name="mazoutais"/><variant name="mazoutait"/><variant name="mazoutant"/><variant name="mazoutas"/><variant name="mazoutasse"/><variant name="mazoutassent"/><variant name="mazoutasses"/><variant name="mazoutassiez"/><variant name="mazoutassions"/><variant name="mazoute"/><variant name="mazoutent"/><variant name="mazoutera"/><variant name="mazouterai"/><variant name="mazouteraient"/><variant name="mazouterais"/><variant name="mazouterait"/><variant name="mazouteras"/><variant name="mazouterez"/><variant name="mazouteriez"/><variant name="mazouterions"/><variant name="mazouterons"/><variant name="mazouteront"/><variant name="mazoutes"/><variant name="mazoutez"/><variant name="mazoutiez"/><variant name="mazoutions"/><variant name="mazoutons"/><variant name="mazoutâmes"/><variant name="mazoutât"/><variant name="mazoutâtes"/><variant name="mazoutèrent"/><variant name="mazouté"/><variant name="mazoutée"/><variant name="mazoutées"/><variant name="mazoutés"/><b>mazouter</b>[mazute]<br><br><ol><li>Polluer par le mazout</li></ol><br>1963 ; de (source<b>mazout</b>)</p></w>

[AlPe]

Thanks gouni, this seems to suggest that Kobo devices simply requires the variants to be coded in the dictionary itself.

(I actually need to check whether I need to modify Penelope to support the variants --- maybe I should anyway.)

[tshering]

Originally Posted by AlPe

Thanks gouni, this seems to suggest that Kobo devices simply requires the variants to be coded in the dictionary itself.

This is right! It works provided the first two characters are not different from the base form.

[gouni]

@Tshering, Alpe This example should be of interest to you, it's the French kobo dictionary.

in red the important things

There are the same word in entry repetiion "name" then in darkorchid the first two letters are different.

I hope that this will bring reflections to help you with new opportunities

être = to be, to exist



<a name="être"/><variant name="seriez"/><variant name="soyez"/><variant name="étiez"/><variant name="êtes"/><variant name="été"/><variant name="fussiez"/><variant name="étaient"/><variant name="soit"/><variant name="êtres"/><variant name="sois"/><variant name="sommes"/><variant name="étant"/><variant name="es"/><variant name="étions"/><variant name="serait"/><variant name="est"/><variant name="serais"/><variant name="serions"/><variant name="fussions"/><variant name="seraient"/><variant name="suis"/><variant name="seront"/><variant name="serons"/><variant name="fusses"/><variant name="soyons"/><variant name="fûtes"/><variant name="étais"/><variant name="fûmes"/><variant name="soient"/><variant name="était"/><variant name="serez"/><variant name="fut"/><variant name="fusse"/><variant name="fus"/><variant name="fussent"/><variant name="furent"/><variant name="fût"/><variant name="seras"/><variant name="serai"/><variant name="sera"/><variant name="sont"/><b>être</b>&nbsp;[ɛtr]<br><br><ol><li>Avoir une réalité, une existence.</li><li>Sert à lier l'attribut, le complément de lieu, de temps, de manière, etc., au sujet.</li><li>S'associe au démonstratif <b>ce</b> pour présenter &qqn;, &qqch;.</li><li>Sert de substitut à <b>aller</b> aux temps composés.</li><li><b><adresse-loc.>En être à</adresse-loc.></b>&nbsp;être parvenu à un certain point, à un résultat.</li><li><b><adresse-loc.>Est-ce que</adresse-loc.></b>&nbsp;</li><li><b><adresse-loc.>Être à</adresse-loc.></b>&nbsp;se trouver, avoir lieu à.appartenir à.indique l'obligation.</li><li><b><adresse-loc.>Être de</adresse-loc.></b>&nbsp;venir defaire partie de</li><li><b><adresse-loc.>Être en</adresse-loc.></b>&nbsp;être vêtu deêtre dans telle situation</li><li><b><adresse-loc.>Être pour</adresse-loc.></b>&nbsp;apporter son soutien, son approbation à.</li><li><b><adresse-loc.>Être sans</adresse-loc.></b>&nbsp;manquer de.</li><li><b><adresse-loc.>N'est-ce pas</adresse-loc.></b>&nbsp;</li><li><b><adresse-loc.>N'être plus</adresse-loc.></b>&nbsp;avoir cessé de vivre.</li><li><b><adresse-loc.>Y être</adresse-loc.></b>&nbsp;être chez soicomprendre</li></ol><br>fin <small><small>IX</small></small>^e s. ; lat. pop. (source&nbsp;<b>*essere,</b>) class. (source&nbsp;<b>esse</b>)</p></w><w><p><a name="être"/><variant name="seriez"/><variant name="soyez"/><variant name="étiez"/><variant name="êtes"/><variant name="été"/><variant name="fussiez"/><variant name="étaient"/><variant name="soit"/><variant name="êtres"/><variant name="sois"/><variant name="sommes"/><variant name="étant"/><variant name="es"/><variant name="étions"/><variant name="serait"/><variant name="est"/><variant name="serais"/><variant name="serions"/><variant name="fussions"/><variant name="seraient"/><variant name="suis"/><variant name="seront"/><variant name="serons"/><variant name="fusses"/><variant name="soyons"/><variant name="fûtes"/><variant name="étais"/><variant name="fûmes"/><variant name="soient"/><variant name="était"/><variant name="serez"/><variant name="fut"/><variant name="fusse"/><variant name="fus"/><variant name="fussent"/><variant name="furent"/><variant name="fût"/><variant name="seras"/><variant name="serai"/><variant name="sera"/><variant name="sont"/><b>être</b>&nbsp;[ɛtr]<br><br><ol><li>La réalité absolue.</li><li>La nature propre, l'essence de</li><li>Le fait d'être</li><li>l'existence</li><li>Ce qui possède l'existence, la vie</li><li>créature</li><li>Individu de l'espèce humaine</li><li><b><adresse-loc.>Culte de l'Être suprême</adresse-loc.></b>&nbsp;culte déiste organisé par Robespierre pendant la Révolution française, en mai-juin 1794.</li><li><b><adresse-loc.>Être de raison</adresse-loc.></b>&nbsp;ce qui n'a d'existence, de réalité que dans notre penséeentité.</li><li><b><adresse-loc.>L'Être suprême</adresse-loc.></b>&nbsp;Dieu.</li></ol><br>v. 1265 ; de (source&nbsp;<b>1. être</b>)</p></w>

[tshering]

Thank you Gauri. I am not sure in what respect the red things are important. I do not understand exactly what you want to demonstrate. (Now, after you have edited the post, I understand better.)
The entry is however an interesting example (any you can find many of this kind) for the fact that many pieces of information of the source dictionary have been included in the kobo dictionary, irrespective whether they have any function in the Kobo environment or not. The only variants that the dictionary engine makes use of are "êtes" and "êtres". The remaining variants are just a waste of space and processing time (at least up to FW 2.1.5; I cannot say anything beyond 2.1.5). These "superfluous" information can of course be useful if you use the data outside of the Kobo reader.

[tshering]

Originally Posted by gouni

I hope that this will bring reflections to help you with new opportunities

Does this mean that all these variants work in the recent FW?

[gouni]

Originally Posted by tshering

Does this mean that all these variants work in the recent FW?

I'm testing on an epub and I give you the results obtained on the version 2.1.5 Kobo Glo.

This presentation of the dictionary kobo version 2.0 does not work with my version 2.1.5 If the first two letters are different. You were right.

[AlPe]

Originally Posted by tshering

The entry is however an interesting example (any you can find many of this kind) for the fact that many pieces of information of the source dictionary have been included in the kobo dictionary, irrespective whether they have any function in the Kobo environment or not. The only variants that the dictionary engine makes use of are "êtes" and "êtres".

I confirm this. In the Italian dictionary there are words which have a structure similar to gouni's example. Moreover, there are completely unuseful variants like the following:

<w>
<p><a name="abaco"/>
<var><variant name="-chi"/></var>
...
</p>
</w>

Here "abaco" (abacus, in Italian) is the singular, while "-chi" means that the plural form is "abachi" (= "abaco" - "co" + "chi"). Clearly the device cannot retrieve this plural form, confirming tshering's observation that in the dictionaries there might be some "extraneous" information, actually not used by the Kobo device.

{For the non-native Italian: the plural forms are written in an Italian dictionary only for those words which do not follow the basic rule -o (s.m.) => -i (pl.m.) or -a (s.f.) => -e (pl.f.), like "bello => belli, bella => belle".}

[Loceka]

Hello,
I already posted that on another thread but I see it does belong here instead :

By combining the method described in this thread to generate Kobo dictionaries and the one of that thread allowing the extraction of .mobi dictionaries, I've wrote a Perl script that creates a Kobo dictionary from an extracted mobi :

Launch it on a linux, using:

perl mobi2kobo.pl -i <input file> -o <output dir>

For the moment it is expecting a cp1252 (WinLatin1) encoded HTML Mobi file as input file. If the input file is UTF8 encoded, it should be changed in the source code.

Also, I noted that some keywords were badly encoded while their definition was correct so I try to estimate whether it happens or not and retrieve the correct keyword. As it is based only on the word length, it may fail (and should therefore be deactivated).

By the time I made the script, I thought the xx.html (all but the 11.html) where reserved to letters only but after reading your posts I see I was mistaken (and the way I did it should be corrected: comparing the uppercase vs the lowercase prefixes).

Still, I noted that if a space character was among the first two letters, the word had to be put in the xa.html instead of the 11.html file.


And I also have a question :
The official Kobo dictionaries seem to have changed a bit and their *.html files now cannot be opened with gzip or 7zip. (you may try on the last released english dictionary)
So have you found a way to open the official .html files ?

Thanks,
Loceka.

[AlPe]

Originally Posted by Loceka

Still, I noted that if a space character was among the first two letters, the word had to be put in the xa.html instead of the 11.html file.

That's an interesting observation, I will test it, thanks.

Originally Posted by Loceka

And I also have a question :
The official Kobo dictionaries seem to have changed a bit and their *.html files now cannot be opened with gzip or 7zip. (you may try on the last released english dictionary)
So have you found a way to open the official .html files ?

I think they are encrypted in some way. Unfortunately I cannot investigate the issue further.

BTW, have you had a look at Penelope, my tool for converting dictionaries? You can find it here: http://www.albertopettarin.it/penelope.html

I might think about integrating read from/write to PRC/MOBI dictionaries, as they seem popular.

[Loceka]

Originally Posted by AlPe

BTW, have you had a look at Penelope, my tool for converting dictionaries? You can find it here: http://www.albertopettarin.it/penelope.html

Actually yes but unfortunately I found it yesterday: after all my researches were done and my converters/scripts coded (XDXF to Kobo and Mobi to Kobo).

Still I may have some use of it because the "official" StarDict converters (makedict and stardict2text) do not handle the stardict dictionaries properly (dictionaries found on the official stardict website...).

Unfortunately I'm not fluent in python so I struggle reading python scripts (but I daresay Perl scripts are way harder to understand for Perl beginners), but I think your script is worth it.

Originally Posted by AlPe

I think they are encrypted in some way. Unfortunately I cannot investigate the issue further.

Well I hope someone will manage it. I'd like to do it myself but I lack the skills to do so.
And, to be honest, their dictionaries are not that bad. Especially their translators which have more entries than the ones I found (even the Oxford en->fr translators) so merging their dictionaries with others would be nice.

[gouni]

Hello all

With you I learned to make dictionary

Stardict == > kobo

I know to do

stardict == > mobi ... with utility pyglossary ( convert a stardict to tab file )and tab2opf + mobigen ( convert tab file to mobi )


I know explode a mobi with mobi-unpack



Can you tell me how to do after mobi for stardict dictionary or xml or tab file?



I have installed python 2.6 for windows and when I have time I'll explore the magnificent work of Alpe.

[AlPe]

Can you tell me how to do after mobi for stardict dictionary or xml or tab file?

Sorry, I do not understand your request.

If it means "How can I obtain a MOBI dictionary from StarDict/XML/CSV file", the answer is: 1) convert your StarDict/XML dictionary to CSV, with tab delimiters and 2) use tab2opf + mobigen.

But I guess that is not your question, because you already knew the answer.

Would you mind stating it clearer, please?

(BTW, I have plans to include reading/outputting from/to MOBI in Penelope, but that is not going to happen before 2 weeks, due to a crazy work schedule.)