[GUI Plugin] WordDumb

[Shark69]

I have opened an issue related to Word Wise for EPUB books in English. Thanks.
But I have a new question. How do you load the JSON dictionary from kaikki?
The one with size 1.4Gb...
If I try to load it from Englihs Wiktionaty maintenance, only the apkg or cvs options are available.

[xxyzz]

Originally Posted by Shark69

But I have a new question. How do you load the JSON dictionary from kaikki?
The one with size 1.4Gb...

Read it line by line: https://github.com/xxyzz/WordDumb/bl...ary.py#L49-L51

Originally Posted by Shark69

If I try to load it from Englihs Wiktionaty maintenance, only the apkg or cvs options are available.

Not sure what you mean...

[Shark69]

Originally Posted by xxyzz

Read it line by line: https://github.com/xxyzz/WordDumb/bl...ary.py#L49-L51


Not sure what you mean...

I mean.... you said:

I use the Wiktionary data from https://kaikki.org, the English JSON file is 1.4GB, other language files should be smaller then that.

I've download it, but I do not how to use it.
In the folder worddumb-lemas I have the file "kaikki.org-dictionary-Spanish.json", but I do not know how to use the file "kaikki.org-dictionary-English.json".
Is it enought to copy it to worddumb-lemas folder?
Because of the code.... it seems it is enought to copy it

[xxyzz]

Why download the json file? The plugin will download and extract it automatically. You just need to click the "custom EPUB" button then select a language, the plugin will handle the rest.

[Shark69]

Originally Posted by xxyzz

Why download the json file? The plugin will download and extract it automatically. You just need to click the "custom EPUB" button then select a language, the plugin will handle the rest.

I thought I was supposed to do it because a I've not any "kaikki.org-dictionary-English.json" file in worddumb-lemmas folder.... after clicking the "custom EPUB" button and selecting English language,

[xxyzz]

The file is deleted after it had been parsed. The extracted file the plugin uses is way more smaller, so the original file is deleted to save space.

[Shark69]

Originally Posted by xxyzz

The file is deleted after it had been parsed. The extracted file the plugin uses is way more smaller, so the original file is deleted to save space.

Thanks....
This is the content of the folder:
Is it correct?

22/07/2022  19:28       413.386.961 kaikki.org-dictionary-Spanish.json
24/07/2022  10:32        19.873.792 kll.en.en.klld
27/07/2022  16:55         1.802.693 lemmas_dump
25/07/2022  11:07       148.931.299 wiktionary_en.json
25/07/2022  11:07       102.244.172 wiktionary_en_dump
22/07/2022  18:56       113.301.832 wiktionary_es.json
24/07/2022  10:35        38.251.415 wiktionary_es_dump

[xxyzz]

Looks fine. As long as everything works, you don't have to worry about these files.

[Shark69]

Everything works like a charm. By the way, I'm daring to make a suggestion I don't know if it is very hard to implement. Have you ever think in generate Word Wise in other languages. I mean, for example... a English book with <ruby> tag in Spanish. Thanks.

[xxyzz]

v3.24.0

New features:

• Import enabled lemmas and difficulty from Anki Deck Package, CSV file and Kindle Vocabulary Builder database
• Add 9854 lemmas and increas difficulty values base on data from Estimating the prevalence and diversity of words in written language by Johns, B. T., Dye, M., & Jones, M. N.

Bug fixes:

• Add pip's hidden option --no-user to disable pip user mode
• Save customize Wiktionary JSON file
• Only show non-English warning when creating Word Wise
• Add missing NER labels for Korean, Swedish and Croatitan spaCy model

[xxyzz]

Originally Posted by Shark69

Have you ever think in generate Word Wise in other languages. I mean, for example... a English book with <ruby> tag in Spanish. Thanks.

kaikki.org only parses the English Wktionary file, so all glosses are in English.

You could keep an eye on https://github.com/tatuylonen/wiktextract/issues/92, and try to use the code to parse the Spanish Wiktionary dump file.

[Shark69]

Originally Posted by xxyzz

kaikki.org only parses the English Wktionary file, so all glosses are in English.

You could keep an eye on https://github.com/tatuylonen/wiktextract/issues/92, and try to use the code to parse the Spanish Wiktionary dump file.

I haven't read it throungly but it seem promising. Thanks

[xxyzz]

Just want you to know I tried the wiktwords command(`wiktwords --language English --out data.json eswiktionary-20220720-pages-articles.xml.bz2`) on the Spanish Wktionary dump file, it runs fine so far but the estimate remain time is more than seven hours and it's still increasing...

I installed the latest commit of wiktextract and wikitextprocessor.

Damn, it's now around ten hours!

[xxyzz]

The command wastes long time paring some useless pages like this one: https://es.wiktionary.org/wiki/Wikci.../000001-010000

[Shark69]

It's incredible. Of course, I appreciate a lot you have been try it. Independent of the end of the story.... Thanks