New dictionary format of firmware 2.14

[mietek81]

Originally Posted by AlPe

Do you mean "merge them into one dictionary for the Kobo" ?

Unfortunately the merging function is not implemented in Penelope yet. Meanwhile, you can do the following:

1) Convert first StarDict dictionary to XML
2) Convert second StarDict dictionary to XML
3) Concatenate the two XML files of 1) and 2) (deleting the "tail/header" at the joining site) into a single XML file
4) Convert the XML file of step 3) into Kobo format

Well.. Yeah, I'm gonna use it on kobo glo.

I had to check in dictionary what does 'Concatenate' means And yes, that's what I would like to do - join them into one single dict.

Basically I should use Penelope to convert those stardict files to XML, then use notepad++ for joining/Concatenating them into single xml and then again convert it to desired format.

So what if I wanted to add another language to english-english dictionary, to have english-english-polish dict? So then I could read english definiton of a word and if that wouldn't help me in understanding it I would have polish translation under english definition. I think somebody have done something like that

[AlPe]

Originally Posted by mietek81

Well.. Yeah, I'm gonna use it on kobo glo.

I had to check in dictionary what does 'Concatenate' means And yes, that's what I would like to do - join them into one single dict.

Basically I should use Penelope to convert those stardict files to XML, then use notepad++ for joining/Concatenating them into single xml and then again convert it to desired format.

Apologies, I use a lot of Latin-based (old-fashioned) English words because I am Italian. You understand correctly. After concatenating the two XML files, you get something like:

<?xml version = "1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE document SYSTEM "dictionary.dtd">
<dictionary>
<entry><key>k1</key><def>d1</def></entry>
<entry><key>k2</key><def>d2</def></entry>
<entry><key>k3</key><def>d3</def></entry>
</dictionary>
<?xml version = "1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE document SYSTEM "dictionary.dtd">
<dictionary>
<entry><key>k4</key><def>d4</def></entry>
<entry><key>k5</key><def>d5</def></entry>
</dictionary>

which should become:

<?xml version = "1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE document SYSTEM "dictionary.dtd">
<dictionary>
<entry><key>k1</key><def>d1</def></entry>
<entry><key>k2</key><def>d2</def></entry>
<entry><key>k3</key><def>d3</def></entry>
<entry><key>k4</key><def>d4</def></entry>
<entry><key>k5</key><def>d5</def></entry>
</dictionary>

{BTW, I now realize that if you export to CSV instead of XML, the editing step might be unnecessary.}

Originally Posted by mietek81

So what if I wanted to add another language to english-english dictionary, to have english-english-polish dict? So then I could read english definiton of a word and if that wouldn't help me in understanding it I would have polish translation under english definition. I think somebody have done something like that

From the Penelope page:

Future versions will include:
- support for merging multiple input dictionaries into a single one, and
- PRC/MOBI support.

[mietek81]

Thank You for all your help and efforts. I will patiently wait for the future versions of this beautiful girl which Penelope is

I had to check the word because im not that fluent in english, my vocabulary is rather poor :P So there's no need for apologies.

Is there a way to send you (I prefer paypal) a small evidence of gratitude for all your work on Penelope?

[AlPe]

You are welcome.

I do not accept money because the code I develop is released "into the world" for free under GPL. Instead, if you want, you can donate to your preferred charity (e.g., Wikimedia Foundation (Wikipedia), EFF, etc.)

[gouni]

My dictionaries no longer work with kobo system 2.5.0

[AlPe]

By "my dictionaries" do you mean "sideloaded" dictionaries, made by Penelope or your own tools, right?

I haven't updated my Kobo (and I plan not to do so), so I cannot confirm directly.

[ShellShock]

Originally Posted by gouni

My dictionaries no longer work with kobo system 2.5.0

Confirmed. Me too. This is a custom dictionary that I have built myself using marisa and 7zip. It was OK with firmware 2.4.

So far I have tried the following to fix it with no success:

Upgraded marisa source from 0.2.0 to 0.2.4.
Various marisa-build options.

This is all random guesses as to what the difference is. May be I'll get lucky and hit on the right combination. It could be the zip options I am using (or not using). In fact I do not use any specific zip options other than the compression format, gzip for the html files and zip for the final file.

[AlPe]

Oh, bad luck, they might have changed the dictionary format.

Does inspecting the Kobo-provided dictionaries yield something?

[ShellShock]

Not that I can tell; the zip contains the same *.html and words files, but as these are binary it is difficult to analyse the differences any further.

[AlPe]

You are right, I forgot about the "encryption".

I will try to have a look at the issue next week. A wild guess is that they changed something in the markup of the HTML files.

(Fun fact: between 3/1 and 4/20, the Google Code page for Penelope got 16 visits from "Toronto, Canada", of which 12 went to clone or download the source code. Clearly, this might well be just a coincidence --- all are from "generic" domains, like "rogers cable inc.".)

[gouni]

Originally Posted by AlPe

By "my dictionaries" do you mean "sideloaded" dictionaries, made by Penelope or your own tools, right?

I haven't updated my Kobo (and I plan not to do so), so I cannot confirm directly.

The two, by penelope and by me.

[ShellShock]

Big brother is watching...? But would it be in their interest to deliberately break custom dictionaries?

[davidfor]

They might think so. But, as Kobo don't actually support custom dictionaries, it is more likely that the change was for some other reason.

[ShellShock]

Interestingly the Kobo English dictionary from firmware 2.1.4 still works on firmware 2.5. Our custom dictionaries use gzipped html and words files, whereas the official dictionaries do not as far as I can tell; if you change the extension to gzip they are not recognised as gzip format. May be this is the crucial difference?

[tshering]

Originally Posted by ShellShock

Interestingly the Kobo English dictionary from firmware 2.1.4 still works on firmware 2.5.

And the new ones work on FW 2.4.

Originally Posted by ShellShock

Our custom dictionaries use gzipped html and words files, whereas the official dictionaries do not as far as I can tell; if you change the extension to gzip they are not recognised as gzip format.

This has been so since FW 2.0.0 or so with some dictionaries.