New dictionary format of firmware 2.14

[ShellShock]

It is certainly technically possible to convert a commercial dictionary to Kobo format, assuming you already own the commercial version and it is for personal use only; even then it may not be legal, but I am not a lawyer so you need to make your own judgement about that. Morally you could argue that you only bought the commercial version so you could format shift it to Kobo format, so it is a win for the publisher - they have sold a copy which they would not have done otherwise. Of course, the resulting Kobo dictionary must not be distributed to anyone else, which would definitely be morally and legally wrong.

[Papi]

Originally Posted by ShellShock

It is certainly technically possible to convert a commercial dictionary to Kobo format, assuming you already own the commercial version and it is for personal use only; even then it may not be legal, but I am not a lawyer so you need to make your own judgement about that. Morally you could argue that you only bought the commercial version so you could format shift it to Kobo format, so it is a win for the publisher - they have sold a copy which they would not have done otherwise. Of course, the resulting Kobo dictionary must not be distributed to anyone else, which would definitely be morally and legally wrong.

I hope there's no problem discussing it, of course I wouldn't distribute the dictionaries (if I ever achieve it), maybe I would give the program to convert dictionaries if that's no problem with the forum rules. Personally I have no problem with trying to convert it since I bought those dictionaries, and I would even avoid the hassle if there was some for sell on the Kobo, unfortunately dictionaries are some of the most evident yet worst implementation in the use of an e-reader generally speaking, not just on the Kobo.

[ShellShock]

Have you seen the instructions attached to the first post in this thread? That gives the steps you need to create a Kobo dictionary, all of which can be automated. There are three main steps:

1. Converting the source dictionary into the html format used by the Kobo.
2. Creating an index from the html.
3. Packaging the html and index into the Kobo dictionary.

Steps 2 & 3 are generic and are good candidates for a generic program; in fact I have code that does this already. I'm thinking about tidying it up and making it generally available; I don't know if there is much demand for it. It is Windows only.

Step 1 is the most time consuming and is bespoke to each source dictionary, as they all have their own formats. Again for this step I write a program, but it is specific to each source dictionary. Depending on the source, it can be done manually using a text editor and e.g., regular expressions, or perhaps xslt tranformations if you have the skill.

I think AlPe is also working on a converter program; see here.

[Papi]

Yep thanks I've read your thread plus this one, thanks to every one's work here it's pretty clear what has to be done, now what I don't know is what the mobipocket dictionary format looks like, but I think it doesn't belong in this thread, I'll do some research now...

[tshering]

Originally Posted by Papi

Also in French we have something very annoying for dictionaries, and it's indeed not handled by the French Larousse dictionary found in the Kobo : s', l', m', t' that can precedes a verb or a noun. For example, abris -》l'abris. If I put l'abris as a variant of abris, as far as I understood, it won't work (as it didn't with go/went).

I cannot see that this should be a major problem. Usually the main entries of dictionaries do not contain, for instance, articles. You would not expect to find "the sun" in an English dictionary, or "le soleil" in a French dictionary. So why would you expect to find "l'abri(s)"? Putting all possible combinations into the dictionary would increase it considerably, as you said:

Originally Posted by Papi

Maybe a file l'.html would work (as shown in the o'clock example), but it would contain a lot if words, basically all the nouns and verbs starting with a vowel.

Therefore, if there is really a need to take care of "l'abri", "s'attarder" and so on this would be better done by the software by removing l', s', qu' and so on while searching. And the KT does this already in a simple way. If you point your finger on "l'heure" it will auto-select "heure" and the correct definition will pop up. I found only a few expressions in the Larousse that would get "lost" in this way, namely "qu'en-dira-t-on", "c'est-à-dire" and "m'as-tu-vu". There might be some more cases, but I think this mechanism works in general rather well. To my mind, adding all l'-expressions and so on would be more pain than gain.

Originally Posted by Papi

... what I don't know is what the mobipocket dictionary format looks like, but I think it doesn't belong in this thread, I'll do some research

If you did not find anything that worked for you you can try converting it with calibre to epub. This will give you one or several (x)html files. However, I did not try it myself.

[Papi]

Thanks for your answer tshering.

Originally Posted by tshering

Therefore, if there is really a need to take care of "l'abri", "s'attarder" and so on this would be better done by the software by removing l', s', qu' and so on while searching. And the KT does this already in a simple way. If you point your finger on "l'heure" it will auto-select "heure" and the correct definition will pop up. I found only a few expressions in the Larousse that would get "lost" in this way, namely "qu'en-dira-t-on", "c'est-à-dire" and "m'as-tu-vu". There might be some more cases, but I think this mechanism works in general rather well. To my mind, adding all l'-expressions and so on would be more pain than gain.

I must have missed something, because I think it should indeed be handled by the software : consider ' to be a word delimiter. But on my glo if I put my finger on l'abris and wait for the dictionary to appear, it will say it cannot find an entrance for "l'abris" so it does consider it to be one word. Maybe I have to resize the selection, maybe I can get rid of the l' by moving the left delimiter.

Originally Posted by tshering

If you did not find anything that worked for you you can try converting it with calibre to epub. This will give you one or several (x)html files. However, I did not try it myself.

Thanks I didn't try that, I tried moby_unpack which seems to work fine, except it does output an html with a big line of approximately 60 Mo (that's for my smallest dictionary). Now I have to decide for the best language to handle that. I think C would be best but it's been a long time since I didn't program in C, maybe some perl or Python but I would have to reformat the file first I think.

[tshering]

Originally Posted by Papi

But on my glo if I put my finger on l'abris and wait for the dictionary to appear, it will say it cannot find an entrance for "l'abris" so it does consider it to be one word.

This is strange. On my Touch, I have really to try hard if I want l' to be selected together with the following word.

[AlPe]

Originally Posted by ShellShock

Have you seen the instructions attached to the first post in this thread? That gives the steps you need to create a Kobo dictionary, all of which can be automated. There are three main steps:

1. Converting the source dictionary into the html format used by the Kobo.
2. Creating an index from the html.
3. Packaging the html and index into the Kobo dictionary.

Steps 2 & 3 are generic and are good candidates for a generic program; in fact I have code that does this already. I'm thinking about tidying it up and making it generally available; I don't know if there is much demand for it. It is Windows only.

Step 1 is the most time consuming and is bespoke to each source dictionary, as they all have their own formats. Again for this step I write a program, but it is specific to each source dictionary. Depending on the source, it can be done manually using a text editor and e.g., regular expressions, or perhaps xslt tranformations if you have the skill.

I think AlPe is also working on a converter program; see here.

Yes, that's correct. I want to add "output to Kobo format" to Penelope, my script that converts XML-like or Stardict dictionaries to the format required by the Bookeen Cybook Odyssey.

I really wanted to spend some time on it last weekend, but my spare time was consumed discussing interesting stuff about (our) Audio-eBooks with DAISY.

I will try to write the output function by the end of this week.

[tshering]

Originally Posted by Papi

But on my glo if I put my finger on l'abris and wait for the dictionary to appear, it will say it cannot find an entrance for "l'abris" so it does consider it to be one word.

Did you try this with several books? Maybe your book uses a sign for an apostrophe that the reader does not recognize as delimiter.

[Papi]

I tried yesterday, out of 4 books, one reacted as expected (' is a delimiter), the 3 others had an apostrophe not considered as a delimiter. Weird, I guess it's not the same exact character, I should check what's inside the file. But I also realized you can adjust the selection to whatever character you want, so it's fine, just a bit more annoying, so I won't try to deal with that at the dictionary level.

[AlPe]

In many cases, eBook typographers uses a typographical apostrophe (U+2019) which renders better than the typewriter apostrophe (U+0027).

See: http://en.wikipedia.org/wiki/Apostrophe#Unicode

[AlPe]

Hi all,

I implemented the output to Kobo format into Penelope, my dictionary-converter tool.

Source code: http://code.google.com/p/penelope-dictionary-converter/
Doc/explanation: http://www.albertopettarin.it/penelope.html

Note that you will probably need Python 2.6+ (not Python 3.x) to run it.

Example:

$ python penelope.py --output-kobo -p bar -f en -t it

Create English-to-Italian dictionary in Kobo format (dicthtml-en-it.zip), from StarDict files bar.*

Example 2:

$ python penelope.py --xml --output-kobo -p bar -f en -t en

Create English dictionary in Kobo format (dicthtml.zip), from XML file bar.xml

===

NOTE 1: probably the management of 11.html entries is not completely correct right now. At the moment every key NOT starting as [a-z][a-z] (when lowercased) will go to 11.html. Better ideas?

NOTE 2: you will need to modify MARISA_BUILD_PATH in penelope.py, pointing it to the directory containing a working build of MARISA.

[AlPe]

=== DELETED, wrong post ===

[ShellShock]

I suspect that 11.html is a "catch-all" file for entires that are not found elsewhere. I have some success with files using characters other than a-z. So I suggest you try it and see.

[AlPe]

If 11.html is a "catch-all", then my current code is good.

Indeed, right now my code does the following: if the lower-cased version of a keyword starts with [a-z][a-z], then it appends that keyword (and its definition) to the corresponding file; otherwise, it appends it to 11.html.

Example:

argon -> ar.html
yoga -> yo.html
a- -> 11.html
-meter -> 11.html
o'clock -> 11.html

My previous doubt was raised by the fact that, in the official Italian dictionary, I found àa.html or wü.html files, suggesting that also accented characters are "allowed" to be there.

For confirming these issues, I will experiment a bit, but now it is quite late...