Size limit of metadata.calibre file?

thundermonkey · 06-26-2022, 10:11 PM

First, thank you to the devs for their hard work on KOReader. I have two jailbroken PW4s, one 6GB and one 32GB. KOReader works great on the 6GB, and is a joy to use.

KOReader on the 32GB also works great until at a certain point - as I add books - the Calibre metadata search feature fails, returning nonsense when I try to browse tags or series. Am I seeing a limit on how large the metadata.calibre file can be?

Some numbers: a 81.5 MB metadata.calibre file caused failure on the 32GB just now. However a 26.3 MB sized file on the 6GB works fine. Furthermore, if I copy that file over to the 32GB then the KOReader metadata search works as expected (although those specific books aren't necessarily there of course).

pazos · 06-27-2022, 03:52 AM

Quote:

Originally Posted by thundermonkey

First, thank you to the devs for their hard work on KOReader. I have two jailbroken PW4s, one 6GB and one 32GB. KOReader works great on the 6GB, and is a joy to use.

KOReader on the 32GB also works great until at a certain point - as I add books - the Calibre metadata search feature fails, returning nonsense when I try to browse tags or series. Am I seeing a limit on how large the metadata.calibre file can be?

Some numbers: a 81.5 MB metadata.calibre file caused failure on the 32GB just now. However a 26.3 MB sized file on the 6GB works fine. Furthermore, if I copy that file over to the 32GB then the KOReader metadata search works as expected (although those specific books aren't necessarily there of course).

Yes. The manual parser is crap code and needs to be replaced. In the mean time you can follow https://github.com/koreader/koreader...ent-1101370638 and change the maximum file size that the program attempts to decode with rapidjson.

Any value there is ok. Currently it is hardcoded to 30MB.

Trying to parse very big files on memory constrained devices will lead to out of memory errors. That was the reason of the crap code in the first place

thundermonkey · 06-27-2022, 10:19 AM

Thanks very much for the quick reply. Increasing MAX_JSON_FILESIZE in metadata.lua as suggested got metadata search working again.

If I hit out of memory limits on the PW4 I'll report back here so people using that device can know what to expect.

06-26-2022, 10:11 PM	#1
thundermonkey Junior Member Posts: 8 Karma: 10 Join Date: May 2022 Device: PW3, PW4, PW5	Size limit of metadata.calibre file? First, thank you to the devs for their hard work on KOReader. I have two jailbroken PW4s, one 6GB and one 32GB. KOReader works great on the 6GB, and is a joy to use. KOReader on the 32GB also works great until at a certain point - as I add books - the Calibre metadata search feature fails, returning nonsense when I try to browse tags or series. Am I seeing a limit on how large the metadata.calibre file can be? Some numbers: a 81.5 MB metadata.calibre file caused failure on the 32GB just now. However a 26.3 MB sized file on the 6GB works fine. Furthermore, if I copy that file over to the 32GB then the KOReader metadata search works as expected (although those specific books aren't necessarily there of course).

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Is there a file size limit on imported books from Kindle for PC?	MorganAndMerlin	Calibre	0	01-15-2022 07:59 PM
limit on size of html file?	hobnail	Sigil	18	11-03-2020 02:21 PM
Touch Is there a maximum epub file size limit?	kefira	Kobo Reader	7	04-12-2015 07:36 PM
RTF file size limit on the Reader?	Aerlock	Sony Reader	11	02-22-2007 04:01 PM

06-27-2022, 10:19 AM	#3
thundermonkey Junior Member Posts: 8 Karma: 10 Join Date: May 2022 Device: PW3, PW4, PW5	Thanks very much for the quick reply. Increasing MAX_JSON_FILESIZE in metadata.lua as suggested got metadata search working again. If I hit out of memory limits on the PW4 I'll report back here so people using that device can know what to expect.