[GUI Plugin] WordDumb

[xxyzz]

I added 9854 lemmas to the lemmas.json file. The difficulty data are from this paper: http://btjohns.com/pubs/JDJ_QJEP_2020.pdf

Excel(33M): http://btjohns.com/JDJ_Prev_supp.xlsx

Commit: https://github.com/xxyzz/WordDumb/co...fb25946211edde

[Shark69]

Originally Posted by xxyzz

I added 9854 lemmas to the lemmas.json file. The difficulty data are from this paper: http://btjohns.com/pubs/JDJ_QJEP_2020.pdf

Excel(33M): http://btjohns.com/JDJ_Prev_supp.xlsx

Commit: https://github.com/xxyzz/WordDumb/co...fb25946211edde

Thanks. The excel file you enclose has 78034 records. Are the previous words retained with difficulty assigned based in the excel file?

[xxyzz]

Originally Posted by Shark69

Thanks. The excel file you enclose has 78034 records. Are the previous words retained with difficulty assigned based in the excel file?

I only deleted one lemma "LED" and increased the difficulty value of some lemmas.

[Shark69]

Originally Posted by xxyzz

I only deleted one lemma "LED" and increased the difficulty value of some lemmas.

Thx. I'll give you my feedback.

[Shark69]

After installing the plugin, I do not see differences in lemmas dump file. Neither there are differences in the Customize Word Wise window. What must I do to use the new functionality?
LED remains in the file.

[xxyzz]

Originally Posted by Shark69

After installing the plugin, I do not see differences in lemmas dump file. Neither there are differences in the Customize Word Wise window. What must I do to use the new functionality?
LED remains in the file.

You need to delete the "lemmas_dump" custom file inside the "worddumb-lemmas" folder to use the default lemmas.

[Shark69]

Originally Posted by xxyzz

You need to delete the "lemmas_dump" custom file inside the "worddumb-lemmas" folder to use the default lemmas.

Then (I've tested it with the Kindle connected too):

calibre, version 6.1.0
ERROR: Excepción no considerada: <b>KeyError</b>:"There is no item named 'lemmas_dump' in the archive"

Traceback (most recent call last):
File "calibre_plugins.worddumb.config", line 187, in open_custom_lemmas_dialog
File "calibre_plugins.worddumb.custom_lemmas", line 44, in __init__
File "calibre_plugins.worddumb.custom_lemmas", line 200, in __init__
File "calibre_plugins.worddumb.utils", line 90, in load_lemmas_dump
File "calibre_plugins.worddumb.utils", line 33, in load_json_or_pickle
File "zipfile.py", line 1503, in open
File "zipfile.py", line 1430, in getinfo
KeyError: "There is no item named 'lemmas_dump' in the archive"

[xxyzz]

You should delete "~/AppData/Roaming/calibre/plugins/worddumb-lemmas/lemmas_dump", this custom dump file "overwrites" the default dump file inside the plugin zip file.

[Shark69]

Originally Posted by xxyzz

You should delete "~/AppData/Roaming/calibre/plugins/worddumb-lemmas/lemmas_dump", this custom dump file "overwrites" the default dump file inside the plugin zip file.

I'd already done it. I'll have to repeat it more carefully because the mistake remains.

[xxyzz]

I guess you deleted the "lemmas_dump" in the code folder then run the calibre-customize command. Install the zip file from GitHub Actions or run the "data/dump_lemmas.sh" will restore that file.

[Shark69]

No way. I completely uninstalled the plugin and inslalled again. The error remains. Bfff... I'll repeat it later.

[xxyzz]

But the error says the plugin zip file doesn't have the "lemmas_dump" file and I have checked the GitHub Actions zip file has it... Otherwise the test jobs can't pass.

[Shark69]

I don't know what I was doing this morning, but once I carefully did it again, it worked perfectly. Thanks.
I have three questions, if you don't mind:
Why are there words in the excel that are not in the Word Wise file? For example, "flash", "outstate", "pluralize"...
Why did you finally decide to use this file and not the one I suggested yesterday? I really don't care which one you use, it's to know your technical opinion. In any case, whatever it is, I think we have won because we have a criterion to level words.
Third:
Are all the words in the excel marked as "true" in Word Wise?

[xxyzz]

Originally Posted by Shark69

Why are there words in the excel that are not in the Word Wise file? For example, "flash", "outstate", "pluralize"...

Kindle can only display Word Wise for lemmas that included in the klld file. Add new lemmas requires more data for the "senses" table: pos_type, full_def, short_def, example_sentence.

Originally Posted by Shark69

Why did you finally decide to use this file and not the one I suggested yesterday?

This paper gathered their data from 26000 books, the paper you posted gathered theirs from testing people. This newer paper not only has more vocabularies but also shows the SD-AP measure has higher accuracy. IMO, their paper is more "reproducible".

Originally Posted by Shark69

Are all the words in the excel marked as "true" in Word Wise?

Only words that included in the klld file are enabled.

[Shark69]

Originally Posted by xxyzz

Kindle can only display Word Wise for lemmas that included in the klld file. Add new lemmas requires more data for the "senses" table: pos_type, full_def, short_def, example_sentence.


This paper gathered their data from 26000 books, the paper you posted gathered theirs from testing people. This newer paper not only has more vocabularies but also shows the SD-AP measure has higher accuracy. IMO, their paper is more "reproducible".


Only words that included in the klld file are enabled.

Thanks. I am testing it and my impresion is very good so far. I'II keep you updated.