Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > KOReader

Notices

Reply
 
Thread Tools Search this Thread
Old 05-10-2022, 08:38 AM   #1
Uncle Robin
Diligent dilettante
Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.
 
Uncle Robin's Avatar
 
Posts: 3,417
Karma: 48736498
Join Date: Sep 2019
Location: in my mind
Device: Kobo Sage; Kobo Libra H2O
KOReader crashs when trying to search Calibre metadata

After finishing a book 20 minutes ago, I decided to "Browse series" for my next book. KOReader crashed. I tried again, it crashed again. So I tried "Browse tags" and it crashed again, tried "browse tags" again, and it crashed again. That' probably why the crash log was too big to attach, hence this Google Drive link - sorry!
Uncle Robin is offline   Reply With Quote
Old 05-10-2022, 12:32 PM   #2
hius07
Wizard
hius07 ought to be getting tired of karma fortunes by now.hius07 ought to be getting tired of karma fortunes by now.hius07 ought to be getting tired of karma fortunes by now.hius07 ought to be getting tired of karma fortunes by now.hius07 ought to be getting tired of karma fortunes by now.hius07 ought to be getting tired of karma fortunes by now.hius07 ought to be getting tired of karma fortunes by now.hius07 ought to be getting tired of karma fortunes by now.hius07 ought to be getting tired of karma fortunes by now.hius07 ought to be getting tired of karma fortunes by now.hius07 ought to be getting tired of karma fortunes by now.
 
Posts: 1,063
Karma: 3581333
Join Date: Sep 2014
Location: Ukraine
Device: Kindle PW2, PW3, PW4, Voyage
Check for the nightly build
https://github.com/koreader/koreader/issues/9085
hius07 is offline   Reply With Quote
Advert
Old 05-10-2022, 12:53 PM   #3
pazos
cosiñeiro
pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.
 
Posts: 1,289
Karma: 2200073
Join Date: Apr 2014
Device: BQ Cervantes 4
Quote:
Originally Posted by hius07 View Post
There're different issues. The other one was due to a refactor in timestamps across the whole project and was easy to fix. This one... I have no idea.

Seems something buried into the UI stack. We need to wait for our setDirty expert @NiLuJe
pazos is offline   Reply With Quote
Old 05-10-2022, 05:44 PM   #4
NiLuJe
BLAM!
NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.
 
NiLuJe's Avatar
 
Posts: 13,482
Karma: 26012494
Join Date: Jun 2010
Location: Paris, France
Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E
Umm, I'm only seeing the crash fixed by #9085 in there? Did I miss one?
NiLuJe is offline   Reply With Quote
Old 05-10-2022, 07:16 PM   #5
Uncle Robin
Diligent dilettante
Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.Uncle Robin ought to be getting tired of karma fortunes by now.
 
Uncle Robin's Avatar
 
Posts: 3,417
Karma: 48736498
Join Date: Sep 2019
Location: in my mind
Device: Kobo Sage; Kobo Libra H2O
Thanks all - the latest nightly has fixed it.
Uncle Robin is offline   Reply With Quote
Advert
Old 05-14-2022, 04:37 AM   #6
pazos
cosiñeiro
pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.
 
Posts: 1,289
Karma: 2200073
Join Date: Apr 2014
Device: BQ Cervantes 4
Huh?. I didn't expect that from that stacktrace.

Anyways. Happy to see it is already fixed
pazos is offline   Reply With Quote
Old 05-21-2024, 02:35 PM   #7
b00kwyrm
Member
b00kwyrm began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Jan 2024
Device: Nook OG 32GB, KT3, PW5 16Gb, KT5, Android KOreader
Sorry to necropost, but I am having this problem too on 2024.04. Where can I find logs to troubleshoot or otherwise share?
b00kwyrm is offline   Reply With Quote
Old 05-21-2024, 02:43 PM   #8
pazos
cosiñeiro
pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.
 
Posts: 1,289
Karma: 2200073
Join Date: Apr 2014
Device: BQ Cervantes 4
Quote:
Originally Posted by b00kwyrm View Post
Sorry to necropost, but I am having this problem too on 2024.04. Where can I find logs to troubleshoot or otherwise share?
https://github.com/koreader/koreader.../bug_report.md
pazos is offline   Reply With Quote
Old 05-27-2024, 01:40 PM   #9
mathil
Evangelist
mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.
 
Posts: 443
Karma: 287725
Join Date: Sep 2014
Location: Europe
Device: Kobo Aura H2O/Glo HD/Libra 2, Kindle PW3/PW5
This has been happening to me with my Kobo Libra 2. My database is, admittedly, quite large (390mb after compression with the KoboUtilities plugin, almost 4000 books on the device). I've been checking github, specifically the latest related bug report here, and am I right in understanding that this issue cannot be solved?
mathil is offline   Reply With Quote
Old 05-27-2024, 02:26 PM   #10
pazos
cosiñeiro
pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.
 
Posts: 1,289
Karma: 2200073
Join Date: Apr 2014
Device: BQ Cervantes 4
Quote:
Originally Posted by mathil View Post
This has been happening to me with my Kobo Libra 2. My database is, admittedly, quite large (390mb after compression with the KoboUtilities plugin, almost 4000 books on the device). I've been checking github, specifically the latest related bug report here, and am I right in understanding that this issue cannot be solved?
Everything can be solved

No idea what's your issue. Follow the bug report template and attach logs (here or in github).

Most likely your calibre json file size is above 30MB and the program switches to a utterly broken mode where it tries to parse json manually, instead of using a proper parser.

That's because the "proper parser" - rapidjson- with its current lua bindings doesn't provide a way to build the result in steps. It tries to build everything in memory until the result is ready.

So the hardcap of 30MiB is there to prevent users with <512MiB or even with +512MiB and high memory pressure to get an out of memory error.

The proper solution is to implement a parser to use for big files, but in the mean time tweaking the hardcap https://github.com/koreader/koreader...ent-1101370638 will work up to the size of your available ram.
pazos is offline   Reply With Quote
Old 05-27-2024, 02:30 PM   #11
pazos
cosiñeiro
pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.
 
Posts: 1,289
Karma: 2200073
Join Date: Apr 2014
Device: BQ Cervantes 4
Another related issue is that, unless you push books using wireless transfers, we consume calibre files as is, with a bunch of metadata we don't care and we don't use.

In wireless transfers we just discard these fields before writting the file, so the json ends up being much more slim.

The difference isn't too much for calibre beginners (up to 10x) but grows a lot if you're one of those advanced users with toons of custom metadata.
pazos is offline   Reply With Quote
Old 05-28-2024, 01:14 PM   #12
mathil
Evangelist
mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.
 
Posts: 443
Karma: 287725
Join Date: Sep 2014
Location: Europe
Device: Kobo Aura H2O/Glo HD/Libra 2, Kindle PW3/PW5
Thank you for your help, pazos - everything is much clearer now. I had already found that link before actually and tried to tweak the hardcap but without understanding what I was doing

My metadata.calibre file is 39.3MB, and changing the max_json_filesize to 40 * etc. doesn't work. The Kobo Libra 2's RAM is 512MB, so I'm not sure it's wise to try and increase that further. And you are correct in assuming that I'm an 'advanced' Calibre user Right now, I'm not willing to change that (or how I send books to the device - I'm still using the Kobo original firmware alongside koreader) so I'm not sure there's much to be done. I'll keep using the regular file search in the meantime.

I would be willing to share the log if you think that's helpful.
mathil is offline   Reply With Quote
Old 05-28-2024, 02:02 PM   #13
pazos
cosiñeiro
pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.
 
Posts: 1,289
Karma: 2200073
Join Date: Apr 2014
Device: BQ Cervantes 4
Make sure those 39.3 are in fact MB and not MiB because that's more than 40x1000x1000 bytes

It should be safe to raise the size to 50 x 1000 x 1000. Do that, restart the program and try again. One of the following two things will happen:

1. The program crashes and you see "./luajit: not enough memory"
2. Everything works.

I hope it is the second, please let me know
pazos is offline   Reply With Quote
Old 05-28-2024, 04:14 PM   #14
mathil
Evangelist
mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.mathil ought to be getting tired of karma fortunes by now.
 
Posts: 443
Karma: 287725
Join Date: Sep 2014
Location: Europe
Device: Kobo Aura H2O/Glo HD/Libra 2, Kindle PW3/PW5
Quote:
Originally Posted by pazos View Post
Make sure those 39.3 are in fact MB and not MiB because that's more than 40x1000x1000 bytes
It's 39.379KB (as shown on Windows), so you're right, more than 40MB. I had to Google that... I think I learned something new today LOL

I did try with 50 x 1000 x 1000 and it works perfectly now. Thank you for your patience
mathil is offline   Reply With Quote
Old 05-31-2024, 10:31 AM   #15
pazos
cosiñeiro
pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.pazos ought to be getting tired of karma fortunes by now.
 
Posts: 1,289
Karma: 2200073
Join Date: Apr 2014
Device: BQ Cervantes 4
Quote:
Originally Posted by mathil View Post
It's 39.379KB (as shown on Windows), so you're right, more than 40MB. I had to Google that... I think I learned something new today LOL

I did try with 50 x 1000 x 1000 and it works perfectly now. Thank you for your patience
Great! Keep in mind you'll need to reapply the change after each update.

Hopefully you won't need to do it many times as a proper fix is in the works: https://github.com/koreader/koreader/pull/11922
pazos is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
koreader deletes calibre metadata after sending book via USB to device Zoasa KOReader 4 06-11-2021 01:38 PM
[Libra] KOReader not updating Calibre metadata Uncle Robin KOReader 9 05-05-2021 01:15 PM
KOReader Calibre Search Adjustment Incanus KOReader 8 02-14-2021 06:46 PM
Calibre: Metadata Search -> Wildcards possible? bigbear77 Library Management 6 11-23-2013 05:44 AM
Ideas on how to improve Calibre: new metadata source (goodreads) + a new cover search MelBr Calibre 2 03-06-2013 03:25 PM


All times are GMT -4. The time now is 08:22 PM.


MobileRead.com is a privately owned, operated and funded community.