New dictionary format of firmware 2.14 - Page 4

tshering · 11-13-2012, 03:07 PM

Why do you think it is using the old index?

AlPe · 11-13-2012, 03:12 PM

Because he was using a "full" English dictionary before adding his own, which had only those two words starting with AA, but in the dictionary search he was seeing more words, exactly the same contained in the previous, "full" dictionary.

It seems like the index ("words") is allocated to the device RAM, so you must force a "flush" when modifying or replacing it, and you can do that by disabling and enabling again the dictionary for that particular language.

ShellShock · 11-13-2012, 04:03 PM

So I went into Definition and switched from English and back again. Now it must be using my index because it is really weird...if I type just "a", it gives me a list of words starting with "a", then if I type the next character, e.g., "aa", the list becomes blank - "0 Results found".

AlPe · 11-13-2012, 04:07 PM

Just to be sure there are no "cache" effects, would you mind trying rebooting your device?

If the problem persists, then something weird is happening with your index/dictionary, which might (or might not) be caused by the fact that you compiled MARISA under windows.

ShellShock · 11-13-2012, 04:08 PM

Quote:

Originally Posted by AlPe

It seems like the index ("words") is allocated to the device RAM, so you must force a "flush" when modifying or replacing it, and you can do that by disabling and enabling again the dictionary for that particular language.

I've just tried that too; it connected via Wifi and did some kind of sync, during which it must have replaced my dicthtml.zip with the official one, because now I am back to the Kobo english dictionary.

tshering · 11-13-2012, 04:13 PM

Quote:

Originally Posted by AlPe

I guess that editing the KoboReader.sqlite file will make them work with arbitrarty names. I do not have my Kobo with me to confirm, though.

I did not try it myself, but clsdclsd did. See the link in my post.

ShellShock · 11-13-2012, 04:14 PM

Quote:

Originally Posted by AlPe

Just to be sure there are no "cache" effects, would you mind trying rebooting your device?

If the problem persists, then something weird is happening with your index/dictionary, which might (or might not) be caused by the fact that you compiled MARISA under windows.

I rebooted using the shutdown button. Now whatever I type in the Definition search textbox, I get "0 Results found", ergo my index is borked. I'm going to play around with marisa to see if I can get it to work under Windows.

I assume my index.txt was OK (see above). I wasn't sure whether just one word per line was correct; there is no reference to the html file which seemed a bit odd.

tshering · 11-13-2012, 04:17 PM

Quote:

Originally Posted by ShellShock

I've just tried that too; it connected via Wifi and did some kind of sync, during which it must have replaced my dicthtml.zip with the official one, because now I am back to the Kobo english dictionary.

I guess you unselected the dictionary at the dictionary management page. AlPe meant to say you should try to choose temporarily another dictionary (e.g. English-French or so).

Edit: Ah, I overlooked some of the previous posts. So this post is somehow unrelated.

AlPe · 11-13-2012, 04:21 PM

Yep, for sure if you leave the "sync" on, you will get "your" dictionary overwritten by the "official" one.

I do not remember exactly and I cannot confirm right now, but try what tshering is suggesting:

1) Switch off the sync
2) Copy your dictionary
3) Switch language to another one
4) Switch language back

You should be done with this.

ShellShock · 11-13-2012, 04:24 PM

Success! What made the difference was to use the -o option with marisa-build, e.g.,

marisa-build -owords src\index.txt

and not pipe the stdout to a file. Using the -o option means that the index file will be in exactly the right format as defined by marisa, and does not rely on Windows redirection of stdout.

Now, I am thinking of converting some public domain dictionaries to Kobo format!

Thanks a lot for your help!

I may write up some instructions for Windows users...

AlPe · 11-13-2012, 04:30 PM

Great!

In case you want to save some time "building" the final dictionary: next weekend, I am going to implement the output to Kobo format into my tool Penelope, initally designed to output dictionary for the Bookeen Cybook Odyssey.

ShellShock · 11-13-2012, 04:38 PM

Cool, about Penelope. I think most my time will be in writing converters to read in existing dictionaries. What input format will Penelope use? Will it be the same as the index.txt and *.html files we have been using so far for the Kobo?

I will probably have to wait until the weekend to do this too; work gets in the way...

AlPe · 11-13-2012, 04:47 PM

Penelope converts FROM StarDict or XML. There is a pseudo-DTD on the project page.

tshering · 11-13-2012, 04:50 PM

Quote:

Originally Posted by ShellShock

So I have built marisa 0.2.0 using Visual Studio 2008.

Could you tell me whether all your marisa tools are working. I compiled them under Windows with mingw and only two tools are working as expected (cf. link), wheras under Linux they all work without any problem. I would like to know whether the compiler makes a difference in this case.

ShellShock · 11-13-2012, 05:30 PM

I ran all the test exes and all the tests passed OK. Also I ran all the tools with the -h option, and they all ran OK. I then ran marisa-lookup against my words index, and it seems to load OK--I certainly have not seen the MARISA_STATE_ERROR you reported. So, it looks like a problem with mingw.

The marisa 0.2.0 source comes with VS2008 projects and they compiled without any real problems. I had two compilation errors the first time I did a Debug build, which were fixed by rebuilding, so I assume there is a small problem with the build order not being 100% correct (i.e., a project depends on the output of another project later on in the build sequence).

I see you can still get the free Visual Studio 2008 Express. Personally I use Development Edition.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
What's file format of dictionary	mnjkl	Kobo Reader	2	12-12-2011 08:48 AM
Dictionary format	jgray	Sony Reader	1	10-25-2010 09:52 AM
English Thesaurus in the dictionary format	osnova	Amazon Kindle	14	12-12-2009 06:42 PM
Dictionary: what version? can it be in firmware?	jedix	Sony Reader Dev Corner	7	12-05-2008 12:00 PM
Webster dictionary in DEPReader format	abigail	Reading and Management	0	08-10-2005 08:00 AM

11-13-2012, 03:07 PM	#46
tshering Wizard Posts: 3,489 Karma: 2914715 Join Date: Jun 2012 Device: kobo touch	Why do you think it is using the old index?

11-13-2012, 03:12 PM	#47
AlPe Digital Amanuensis Posts: 727 Karma: 1446357 Join Date: Dec 2011 Location: Turin, Italy Device: Several eReaders and tablets	Because he was using a "full" English dictionary before adding his own, which had only those two words starting with AA, but in the dictionary search he was seeing more words, exactly the same contained in the previous, "full" dictionary. It seems like the index ("words") is allocated to the device RAM, so you must force a "flush" when modifying or replacing it, and you can do that by disabling and enabling again the dictionary for that particular language.

11-13-2012, 04:03 PM	#48
ShellShock Wizard Posts: 1,178 Karma: 2431850 Join Date: Sep 2008 Device: IPad Mini 2 Retina	So I went into Definition and switched from English and back again. Now it must be using my index because it is really weird...if I type just "a", it gives me a list of words starting with "a", then if I type the next character, e.g., "aa", the list becomes blank - "0 Results found".

11-13-2012, 04:07 PM	#49
AlPe Digital Amanuensis Posts: 727 Karma: 1446357 Join Date: Dec 2011 Location: Turin, Italy Device: Several eReaders and tablets	Just to be sure there are no "cache" effects, would you mind trying rebooting your device? If the problem persists, then something weird is happening with your index/dictionary, which might (or might not) be caused by the fact that you compiled MARISA under windows.

11-13-2012, 04:21 PM	#54
AlPe Digital Amanuensis Posts: 727 Karma: 1446357 Join Date: Dec 2011 Location: Turin, Italy Device: Several eReaders and tablets	Yep, for sure if you leave the "sync" on, you will get "your" dictionary overwritten by the "official" one. I do not remember exactly and I cannot confirm right now, but try what tshering is suggesting: 1) Switch off the sync 2) Copy your dictionary 3) Switch language to another one 4) Switch language back You should be done with this.

11-13-2012, 04:24 PM	#55
ShellShock Wizard Posts: 1,178 Karma: 2431850 Join Date: Sep 2008 Device: IPad Mini 2 Retina	Success! What made the difference was to use the -o option with marisa-build, e.g., marisa-build -owords src\index.txt and not pipe the stdout to a file. Using the -o option means that the index file will be in exactly the right format as defined by marisa, and does not rely on Windows redirection of stdout. Now, I am thinking of converting some public domain dictionaries to Kobo format! Thanks a lot for your help! I may write up some instructions for Windows users...

11-13-2012, 04:30 PM	#56
AlPe Digital Amanuensis Posts: 727 Karma: 1446357 Join Date: Dec 2011 Location: Turin, Italy Device: Several eReaders and tablets	Great! In case you want to save some time "building" the final dictionary: next weekend, I am going to implement the output to Kobo format into my tool Penelope, initally designed to output dictionary for the Bookeen Cybook Odyssey.

11-13-2012, 04:38 PM	#57
ShellShock Wizard Posts: 1,178 Karma: 2431850 Join Date: Sep 2008 Device: IPad Mini 2 Retina	Cool, about Penelope. I think most my time will be in writing converters to read in existing dictionaries. What input format will Penelope use? Will it be the same as the index.txt and *.html files we have been using so far for the Kobo? I will probably have to wait until the weekend to do this too; work gets in the way...

11-13-2012, 04:47 PM	#58
AlPe Digital Amanuensis Posts: 727 Karma: 1446357 Join Date: Dec 2011 Location: Turin, Italy Device: Several eReaders and tablets	Penelope converts FROM StarDict or XML. There is a pseudo-DTD on the project page.

11-13-2012, 05:30 PM	#60
ShellShock Wizard Posts: 1,178 Karma: 2431850 Join Date: Sep 2008 Device: IPad Mini 2 Retina	I ran all the test exes and all the tests passed OK. Also I ran all the tools with the -h option, and they all ran OK. I then ran marisa-lookup against my words index, and it seems to load OK--I certainly have not seen the MARISA_STATE_ERROR you reported. So, it looks like a problem with mingw. The marisa 0.2.0 source comes with VS2008 projects and they compiled without any real problems. I had two compilation errors the first time I did a Debug build, which were fixed by rebuilding, so I assume there is a small problem with the build order not being 100% correct (i.e., a project depends on the output of another project later on in the build sequence). I see you can still get the free Visual Studio 2008 Express. Personally I use Development Edition.

Advert

Advert