Clara crashes on epub opening

[Barruel]

Hi.

I have a problem with my Clara I can't seem to be able to fix.

Normally I read kepubs, but lately I'm playing with fonts and wanted to open some documents in the epub format, which handles typesetting better than Kobo's engine used to display kepubs. So I sideloaded them to the device without converting to kepub (by disabling "Extended Kobo features" in the Calibre Kobo Touch Extended driver plugin) and I found that my Clara crashes everytime I try to open an epub file.

Same thing happens with plain html files, which seem to be handled also by RMSDK.

Some background: I have a fairly large number of books and my DB is over 700 MB. This probably doesn't matter because I've hidden my books folder with a dot and placed a fresh vanilla database I had backed up and my device still can't open epubs.

I'm running firmware 4.26.16704 and I'm using kobopatch with the following kobopatch.yaml:

## Works with kobopatch v0.15.0 and later.
## You can update kobopatch by downloading the latest release from https://github.com/pgaskin/kobopatch/releases. 
version: 4.26.16704
in: src/kobo-update-4.26.16704.zip
out: out/KoboRoot.tgz
log: out/log.txt

patchFormat: kobopatch

patches:
  src/nickel.yaml: usr/local/Kobo/nickel
  src/libadobe.so.yaml: usr/local/Kobo/libadobe.so
  src/libnickel.so.1.0.0.yaml: usr/local/Kobo/libnickel.so.1.0.0
  src/librmsdk.so.1.0.0.yaml: usr/local/Kobo/librmsdk.so.1.0.0

## You can put lines in the following section to override the enabled state of patches.
## The indentation matters! Each override should be indented by 4 spaces. Add to the 
## section below. This section can be copy and pasted into newer patch versions to
## keep your selections.
##
## Example of how it should look:
## overrides:
##   src/nickel.yaml:
##     Custom synopsis/details line spacing: yes
##     Whatever the yaml is called: no
##   src/libadobe.so.yaml:
##     You get the idea: yes
overrides:
  src/nickel.yaml:
    Reduce top/bottom page spacer: yes
    Custom synopsis details line spacing: yes
    Dictionary pop-up - increase available text area: yes
##	Change pop-up footnote font-family
    Reduce new header/footer height: yes
    Custom collection/author header title font: yes
    Custom header/footer captions: yes
    Remove footer (row3) and increase cover size on new home screen: yes
    Remove footer (row3) on new home screen: yes
    Change TOC level indentation: yes
  src/libadobe.so.yaml:
  src/libnickel.so.1.0.0.yaml:
    My 24 line spacing values: yes
    Custom left & right margins: yes
    Custom font sizes: yes
    ePub disable built-in body padding-bottom: yes
    Custom kepub default margins: yes
    Set KePub hyphenation: yes
    Force user line spacing in KePubs: yes
    Force user line spacing in ePubs (part 1 of 2): yes
    Force user font-family in ePubs (Part 1 of 2): yes
    Un-Force user text-align in div,p tags in KePubs: yes
    Un-Force user font-family in KePubs: yes
    KePub stylesheet additions: yes
    ePub uniform font scale: yes
##    Set font scale factor: yes
    Dictionary text font-family/font-size/line-height: yes
    Always show confirmation dialog before upgrading: yes
    Remove recommendations (row1col2) from home screen: yes
    Allow showing info panel on random screensaver: yes
    Remove title from reading header/footer: yes
    Don't uppercase header/footer text and change page number text: yes
    Enable advanced settings for all fonts: yes
    Customize ComfortLight settings: yes
    FeatureSettings - MyWords: yes
    FeatureSettings - ExportHighlights: yes
    DeveloperSettings - AutoUsbGadget: yes
    PowerSettings - UnlockEnabled: yes
  src/librmsdk.so.1.0.0.yaml:
##    Disable orphans/widows avoidance: yes
    Force user line spacing in ePubs (Part 2 of 2): yes
    Force user font-family in ePubs (Part 2 of 2): yes

## TRANSLATIONS ##
# Optional, use only if lrelease is not in PATH and if translations are needed
# lrelease: /path/to/lrelease

# Uncomment the following to add translations (replace lc with the language code)
# translations:
#   src/whatever.ts: usr/local/Kobo/translations/trans_lc.qm

## ADDITIONAL FILES ##
# Uncomment the following to add additional files to the tgz (like init scripts or hyphen dicts)
# The files will be root-owned, and world readable, writable, and executable (0777)
# files:
#   src/whatever.txt: usr/local/Kobo/whatever.txt
#   src/whateverToPutInMultiplePlaces.txt:
#     - usr/local/Kobo/location1.txt
#     - usr/local/Kobo/location2.txt

I'm also running the package with KFMon, KOReader and Plato available here at MobileRead.

Any ideas about what is preventing my device to open epubs? This is driving me nuts. Can I access crash logs anywhere?

[rtiangha]

What do you mean by "crash"? As in the device reboots? Turns off?

If it reboots, that's probably the watchdog timer forcing a reboot because RMSDK is taking too long to open the file and thinks something is wrong. This has happened to me with large databases and basically boils down to running low on RAM.

One thing you could try is rebooting the device to free up RAM, then immediately open the book once you're logged in. Else, sometimes it'll open if you wait long enough, but unfortunately, sometimes sickel won't let you. There's a patch that can increase the timeout to 30 seconds, but I don't know if it was updated for firmware 4.26. If that's not long enough, you can also disable sickel entirely (will need to be reflashed after every firmware update); it may take 2 minutes to open the file, but it might open if you wait long enough. If it takes too long, you can always hard power off.

Otherwise, no idea. I've started using koreader or plato for things in epub format and just use nickel for kepubs.

[Barruel]

Thanks, rtiangha.

Disabling Sickel helped a bit. My Clara tried to open the books. It stayed with a blank screen forever, but it didn't reboot, so I guess it was Sickel triggering the reboots, which is what I previously referred as "crashes".

It looks like it was indeed a RAM issue. I had 1400+ sideloaded font files. I trimmed that to ~900 and now I can load epubs.

It's a bit strange. Nickel, KOReader and Plato can all load books in those conditions. I expected RMSDK to be able to do the same thing.

[rtiangha]

Yeah, the default sickel timeout is something ridiculously low like 5 or 10 seconds. I wish it was more configurable. That file may have opened eventually if you waited long enough (like 5+ minutes).

I think part of the problem is that nickel loads all user fonts into RAM as soon as you open a book (which is why rebooting first before opening a RMSDK file sometimes works). And if your database is already larger than what RAM the device ships with, it just compounds the issue since now it has to continuously swap to a pagefile on a really slow SD card. I also had to cut down fonts to get things working once my database got to a certain size, but eventually, I just ended up using other reader programs to read things that weren't kepubs. kepubs work fine since nickel is loaded into RAM on boot; RMSDK is not and the free RAM usually gets eaten up by other things first over time.

But 1,400+ font files is a lot, lol.

[davidfor]

Originally Posted by Barruel

Thanks, rtiangha.

Disabling Sickel helped a bit. My Clara tried to open the books. It stayed with a blank screen forever, but it didn't reboot, so I guess it was Sickel triggering the reboots, which is what I previously referred as "crashes".

It looks like it was indeed a RAM issue. I had 1400+ sideloaded font files. I trimmed that to ~900 and now I can load epubs.

Isn't that unwieldy? It means that there are somewhere between 250 and 900 extra lines in the fonts menu. How can you find one to use?

It's a bit strange. Nickel, KOReader and Plato can all load books in those conditions. I expected RMSDK to be able to do the same thing.

Not strange at all. All four applications are written by different people with different requirements and different expectations. The idea that all four would work exactly the same is a lot stranger than that one wouldn't. I was told by a Kobo developer that the RMSDK was inefficient in how it loads fonts, though I would not have thought it would load all the fonts. But, maybe it is loading them to get the details so they can be used when needed. That would make sense with a sickel timeout reboot and it working when the number of fonts was reduced.

And for the record, "nickel" is the application being run for everything. It is the UI to navigate to get to the books. The kepub render was based on ACCESS. That has changed its name, but, I've lost track of what it is now called.

[davidfor]

Originally Posted by rtiangha

Yeah, the default sickel timeout is something ridiculously low like 5 or 10 seconds. I wish it was more configurable. That file may have opened eventually if you waited long enough (like 5+ minutes).

No, the sickel timeout is 60 seconds.

[Barruel]

Originally Posted by davidfor

Isn't that unwieldy? It means that there are somewhere between 250 and 900 extra lines in the fonts menu. How can you find one to use?

I have such number of fonts just because I was testing fonts. I don't plan to do that permanently.

I don't use the menu to change the fonts. KOReader can generate a "test font document" which contains some text which is set to show in every single font installed in the device. This document is in plain HTML format. It is the document RMSDK failed to open.

I trimmed the text to fit in one screen, imported the document into Calibre and converted it to ebook format. This way I just go to the next page to see the next font. Don't need to use the menu at all. I just set the font to "Publisher default".

The kepub I generated works ok in Nickel, but RMSDK couldn't handle the epub. I was interested in seeing the RMSDK output because Nickel is pretty bad at handling kerning and it does funny things with most fonts when displaying the "[leoˈnardo diˌsɛr ˈpjɛːro da (v)ˈvintʃi]" part of the text. See the difference between Baskerville 10 Pro and IBM Plex Serif in the attached screenshots.

BTW, Calibre and the rendering engines in the devices mess it up when the font name contains a word made solely of numbers. Calibre converts every single

<div style='font-family: whatever'>

declaration into a CSS style, but for some reason the numbers are ignored if they are not attached to a letter. I had to manually fix "Aldine 721 BT", "Baskerville 10 Pro" or "Coranto 2" and a few others in the epub by enclosing the font name in quotes. There where just a few, luckily.

Originally Posted by davidfor

I was told by a Kobo developer that the RMSDK was inefficient in how it loads fonts, though I would not have thought it would load all the fonts. But, maybe it is loading them to get the details so they can be used when needed. That would make sense with a sickel timeout reboot and it working when the number of fonts was reduced.

Very interesting info. It explains why RMSDK struggles so much with that document. It seems that it loads all the fonts regardless of whether they are used in the document it is handling or not, because I couldn't load any epub at all.

I guess I can just test the fonts in RMSDK in small batches, so problem solved. The problem I have now is to keep just 50 fonts or so. There are very very good fonts out there.