Pocketbook dictionary

[snsdg]

Originally Posted by datyoma

Hi, there are a few more options for converting now:

- https://github.com/ilius/pyglossary/releases/tag/5.3.0 (I added *.dic output plugin literally this week)
- https://datyoma.codeberg.page/pbdt/v1/ - an experimental converter from StarDict/XDXF only; everything happens in browser, no server side

Hi, datyoma!

First of all, a huge thank you for the effort you’ve put into this.

I tried using your online converter to convert dictionaries from .DIC format to StarDict format. The conversion process works successfully for most dictionaries. However, after downloading the converted dictionary, it turns out the file with the .dz extension has a CRC error and is essentially unreadable. Any attempt to extract the .dict file contained within it results in an error across all archiving tools. So, my question is: could you fix this issue or disable the compression (i.e., have the .dict file downloaded in .dict format rather than .dict.dz)?

Thanks in advance for your help and for your reply!

[datyoma]

@snsdg,

Thanks for testing! There was a bug in checksum calculation, which is fixed now - please try the online converter again.

[snsdg]

Thanks for the reply!

It works almost perfectly now. The error persists to some extent: the .dz file can now be decompressed and read by GoldenDict, but then a "DICTZIP error: inflate: (null)" still occurs with some words (though not all). I tried decompressing the .dict file contained within the .dz archive and reading it directly in GoldenDict, and that worked perfectly. Then, I decided to try compressing the .dict file again using the online version of DictZip (I packed the .idx files too), and everything works fine.

Thanks for the excellent work - I’m really happy with it, and it will be useful for many other people too!

Since I maintain a cloud storage repository for dictionaries, I will gradually start converting more old dictionaries for PocketBook as well, and if I notice any further bugs, I will post again.

Huge thanks once again!

[snsdg]

Another error occurred: during the conversion of a large file in XDXF format (366 MB), an "unreachable executed" error appeared, and consequently, the conversion was unsuccessful. For a few files, the converted dictionary has a size of zero, but I assume this is due to errors in the structure of the input dictionary, not the online converter.

[datyoma]

I deployed another fix for .dict.dz - please check whether it works now.

As for large XDXF files, the online converter was never designed for this, the largest size I tested it with was about 50MB. In general, if you want to process multiple files, it's better to use the CLI version (https://codeberg.org/datyoma/pbdt) so that you can write a script to loop over files.

[snsdg]

It turns out it’s not just large files. Go to that folder - specifically the RUM-RUM subfolder - and download the file `dex09-2009_reconstructed_Oct6th2022.dic` (16.4 MB); it triggers the same "unreachable executed" error:

https://e1.pcloud.link/publink/show?...WB2RAb2Gmg3X#/

In my opinion, it is quite possible that the error has nothing to do with the file size.

The "DictZipError" seems to have been resolved - at least, it didn't appear again during testing.

Congratulations, and thanks once again for the lightning-fast response!

[datyoma]

Lucky you, GPT 6.1 Sol was released yesterday evening, so I asked it to invent an intermediate in-memory format with compressed blocks and an index (not unlike StarDict but with lz4 instead of gzip) and convert input to that format in a streaming fashion, and then also benchmark memory usage and performance against the baseline. I deployed it to https://datyoma.codeberg.page/pbdt/v2-alpha/ but the source code is too horrible to publish. I still recommend using the CLI option though.

[snsdg]

Thanks again!

The Romanian one converted without any issues this time. The other one I mentioned couldn't be converted - it throws the same old error - but it’s readable by GoldenDict and in XDXF format anyway. There are a few other stubborn ones, but I suspect they are encrypted (since I ripped them from Windows dictionaries) or are actually in text format despite their file extension and weren't originally created for PocketBook.

If you intend to keep dealing with me and my requests and to keep making experiments with your code, I can provide links as well. I generally don’t need them anyway, and I only require them for cloud storage - it’s good if they’re there, but it’s not a problem if they aren’t.

Once again, thank you for the tremendous effort you have put in!

The dictionaries are here:

1. The Remede's (in XDXF format; remede.xdxf is the problematic file):

https://github.com/camarm-dev/remede/releases

2. On WinEdt.org:

https://www.winedt.org/dict.html and https://www.winedt.org/dictASCII.html

Here - in section "Plain Text":

https://ktranslator.sourceforge.net/dictionaries.html

3. Ripped from 3 Bulgarian programs:

https://www.mediafire.com/folder/tly...b/Experimental

...and that is finally everything I have at the moment.

They all return the error "failed to load Pocketbook: invalid dictionary signature".