Index of Custom Dictionaries for Kobo eReader

[oren64]

Originally Posted by AlPe

@oren64: I found and (hopefully) fixed the problem with outputting to a relative path, I pushed a new version on GitHub and PyPI, v3.1.0. You can upgrade penelope with:

> pip install penelope --upgrade

It generate dicthtml file in the folder.
it didn't delete the tmp... files in C:\...\local\Temp.

[AlPe]

It generate dicthtml file in the folder.

Good.

it didn't delete the tmp... files in C:\...\local\Temp.

Are you using the -k flag (or --keep) ? If so, that's the expected behavior.

If you not, then that's a (Windows) bug. I cannot reproduce it in Linux.

It is quite strange, as I use the Python built-in tempfile module to create the temporary directories and files, and they are supposed to be removed automatically (EDIT for clarity: by my code) as soon as the execution ends (without errors). Are you sure that the tmp* directories you are seeing are not coming from previous executions gone bad? Just delete all of them, run penelope again, and see if a new tmp* directory persists after the execution ended.

[oren64]

Without the -k flag it's only deleting the folder with the files(d.dict,d.ifo...), and folder with the HTML files not deleting.
despite that there is a ([DEBU]delete temp dir...), maybe you need to delete the folder after the zip file is created, but it's not a big deal to me.

[AlPe]

OK, got it. It appears to be a issue with shutil.rmtree() (the function that is supposed to remove the temporary directory) on Windows. I will investigate.

For now, the only possible workaround consists in manually purging the Temp directory.

[AlPe]

Uploaded penelope 3.1.1(.1) to PyPI, including a workaround for forcing the removal of temp directories on Windows. Try upgrading:

> pip install penelope --upgrade

and see it works for you.

[oren64]

Almost it delete the HTML files except to 21 files.

Edit:files belong to English-Hebrew dictionary, I try with English dictionary and the folder deleted.

[AlPe]

Woah... now it seems that the problem is that those file names contain accented (i.e., not ASCII-7) letters. I guess I will need to dig the Python os documentation and hacks...

[embryo]

Wellcome to the Python's 2.x unicode nightmare...
The way I do it, is to rename all the temp filenames to ascii - do what I want with them and rename them back to the original name.

[AlPe]

Since v3.0, Penelope works with both Python 2.7.x and 3.x.

It looks more like a shell encoding problem.

I tried reproducing oren64's issue on a Windows Vista machine (I do not have a more recent Windows machine), and with Python 2.7.x I was not able to, penelope v3.1.1 worked as expected, including cleaning up all the temporary files. I also tried changing the windows prompt encoding using chcp and it still worked as expected, also with ??.html files containing non-ASCII characters.

Then, I thought the bug was with Python 3.5 only. I tried installing it on the same Vista machine, but I was not able to run it since it complains about a missing DLL, and I gave up on it.

Yeah, maybe the workaround "rename the ??.html files to say 99.html before removing them" works, but I would like to be able to reproduce the issue and understand it, before throwing in random hacks.

@oren64: sorry, but for now you need to clean your Temp manually. What is the output of

> chcp

(It should print the current shell encoding)

Can you try changing it to latin1 and try again?

> chcp 850

Have you tried using Python 2.7.x instead?

[embryo]

The killer combination is Python 2.x on Windows trying anything with command line (popen).
I think its a known bug, but they don't fix it since they don't really support 2.x versions anymore...
... or at least that was my conclusion when I had hit the same wall.
Any other solution would be really appreciated.

[oren64]

Originally Posted by AlPe

@oren64: sorry, but for now you need to clean your Temp manually. What is the output of

> chcp

(It should print the current shell encoding)

Can you try changing it to latin1 and try again?

> chcp 850

Have you tried using Python 2.7.x instead?

I am using Python 2.7.10, it's only 5kb and not 5-10mb so it okay, and it not happen on all dictionaries, i try with English dictionary and folder deleted.

I don't understand what to do with "chcp" where and how do i use it?

[embryo]

Originally Posted by oren64

I don't understand what to do with "chcp" where and how do i use it?

Just open a command prompt (search for cmd.exe) type chcp and hit enter.
It tells you what is the code page that the terminal is using.
To change it, type chcp, the desired code page number and hit enter.

[oren64]

Originally Posted by embryo

Just open a command prompt (search for cmd.exe) type chcp and hit enter.
It tells you what is the code page that the terminal is using.
To change it, type chcp, the desired code page number and hit enter.

my chcp is 862: OEM Hebrew; Hebrew (DOS).

I tried all the Latin i can find 850,858,852,1252... , and it didn't work.
if i exit cmd.exe and open it again it go back to 862.

EDIT: I tried to install penelope on 3.4.3 and get an error.

Originally Posted by AlPe

If the resulting dictionary works on your Kobo, you are good. Otherwise, try:

2. --group-by-prefix-length=1 --group-by-prefix-merge-min-size=1000 --group-by-prefix-merge-across-first

And again see if that works on your Kobo.

I try making the Chinese-Russian Dictionary and it create a zip file, but it also didn't delete all the HTML files in Chinese.

[AlPe]

Now I am a bit confused.

Forget about the issue with the Temp/ files not being deleted, and the fact that there is an error installing Python 3.x.

I understand the following:

1. You installed Python 2.7.x and penelope
2. Using Python 2.7.x, you generated the dictionary you attached in the previous post
3. There are some files in Temp/ that are not deleted by Penelope

Now:

A. Is the dictionary file output in 2. working? If so, you are good.

B. About issue 3. : just delete them manually or periodically use the Windows "cleanup" tool (I do not know how it is called now --- it deletes all the temporary directories in the system).

I understand it is inconvenient, but I cannot fix the issue if I cannot reproduce it:

i) On my Vista machine,
ii) with Python 2.7.x and
iii) penelope v3.1.1 and
iv) the non-ASCII-files you provided a few posts ago and
v) chcp=850,

my temporary files got deleted as expected.

(In the weekend I will try using chcp=862 and see if I can reproduce the bug.)

[oren64]

Originally Posted by AlPe

1. You installed Python 2.7.x and penelope
2. Using Python 2.7.x, you generated the dictionary you attached in the previous post
3. There are some files in Temp/ that are not deleted by Penelope

1 & 2) yes, it create the dicthtml.zip in the folder and i used 2.7.10.
3) it didn't delete the Temp folder in C:\...\lokal\temp, with Html files in Chinese (141 files), it only deleted (words,1.html, b.html).

I also tried Hebrew-Hebrew dictionary and it delete the temp folder.

B. About issue 3. : just delete them manually or periodically use the Windows "cleanup" tool (I do not know how it is called now --- it deletes all the temporary directories in the system)

I find CCleaner software.