Library search

[ctop]

Dear Koreader users and developers,
I started using Koreader about a year ago, when I got myself a Likebook Ares. Initially, I had some problems adjusting to the way Koreader deals with my fairly large library, but finally I found a way to use it to my satisfaction, with a combination of search using calibre metadata and tags defined with calibre before sending to the device.
This stopped working some days ago, and since the device had a system update, I used this opportunity to upgrade the whole system, so now I am finally using the latest Koreader version, 2021-01 again. There are many nice changes and the wireless calibre connection works wonderfully.
However, when I start using the "Find a book via calibre metadata" function as before, the system will hang for a while (which is not surprising, even before it would take a while to load the library after a fresh start of the program), but then finally crash.
So I have no idea what is wrong and how to fix it and am without the ability to use the search function! This makes it very difficult to go to a different book.
Is there any way I can solve this problem, or shed light on where things go wrong? Any help appreciated.
Ctop

[NiLuJe]

How many books are we talking about here?

(c.f., https://github.com/koreader/koreader/pull/7159, which should have landed in today's nightly).

[ctop]

Originally Posted by NiLuJe

How many books are we talking about here?

(c.f., https://github.com/koreader/koreader/pull/7159, which should have landed in today's nightly).

About 5000 in the internal storage area and another 20000+ on the sdcard. The filesize of the metadata file for the internal library is 10.9 MB, the one on the sdcard is 131.4 MB.

All the best,
Ctop

[pazos]

Originally Posted by ctop

About 5000 in the internal storage area and another 20000+ on the sdcard. The filesize of the metadata file for the internal library is 10.9 MB, the one on the sdcard is 131.4 MB.

All the best,
Ctop

All versions from 2020.05 to 2021.01 will have a hard time decoding such large metadata files. The library used for that, rapidjson-lua, does the decoding on a single step and can easily eat +10x the size of the file while decoding.

Use a recent nightly, which should behave like versions prior to 2020.05 for files bigger than 30MB

[ctop]

Originally Posted by pazos

All versions from 2020.05 to 2021.01 will have a hard time decoding such large metadata files. The library used for that, rapidjson-lua, does the decoding on a single step and can easily eat +10x the size of the file while decoding.

Use a recent nightly, which should behave like versions prior to 2020.05 for files bigger than 30MB

Ah, that is great to know, I will give that a try! Thanks a lot, Ctop

(a few hours later)
I can now confirm that the latest nightly build indeed solves this problem! Great!!