Loading many pdfs via ssh on paperwhite eventually fails

[jonvarteresian]

i'm trying this now:

"/usr/local/bin/indexerdisable.sh" should disable indexer for quite some time. i can send it periodically

[jonvarteresian]

so i paused the indexer and it still hung after 900 displays.

This is a little more interesting, and here is what I did.

Paused indexer
loaded 2 pdfs into /mnt/us/documents as described above

i then load them in the viewer alternately with

lipc-set-prop com.lab126.appmgrd start file://<path><fn>


everything works perfectly until i hit around 900 cycles then things get very slow and then stop updating. lipc then starts with 'unable to open application'.

so it has nothing to with the ssh transfer because i only transferred 2 files and it has nothing to do with the indexer because i can see in the messages log that the indexer is paused.

it must have to do with the fact that i open the pdf viewer 900+ times. Either it needs to be closed in a specific way (which i am not doing) or there is a memory leak (which i dont see).

[NiLuJe]

FWIW, in KUAL, we kill ourselves via lipc-set-prop com.lab126.appmgrd stop app://com.lab126.booklet.kindlet or lipc-set-prop com.lab126.appmgrd stop app://com.mobileread.ixtab.kindlelauncher in the Booklet variant.

Alternatively, a more generic lipc-set-prop com.lab126.appmgrd backward 0 might be enough, depending on how the app is registered.

[jonvarteresian]

Originally Posted by NiLuJe

FWIW, in KUAL, we kill ourselves via lipc-set-prop com.lab126.appmgrd stop app://com.lab126.booklet.kindlet or lipc-set-prop com.lab126.appmgrd stop app://com.mobileread.ixtab.kindlelauncher in the Booklet variant.

Alternatively, a more generic lipc-set-prop com.lab126.appmgrd backward 0 might be enough, depending on how the app is registered.

well,

i added a lipc-set-prop com.lab126.appmgrd backward 0 which nicely closes the pdf viewer and returns me to the home screen before loading another pdf, but the same problem still happens.

looking at process usage i can see that dbus-daemon is pegged at 97%, so that must be where the problem lies.

I poked around but there doesnt seem to be a way to restart the dbus-daemon on the kindle. Still looking

[NiLuJe]

Huh. lipc is talking over dbus, of course, so, that at least makes a tiny bit of sense, but, still, that's weird .

It's probably handled by one of the upstart jobs, but it might be a fairly early boot stuff, so it might take down the full framework with it (meaning you'll have to sit through the ~50s framework startup, even a "warm" restart like that).

EDIT: Yep, and it's aptly named "dbus" ;D.

And it will indeed take X down with it, which in turn brings down lab126_gui .

EDIT²: Assuming your profile is accurate, of course, as upstart is FW 5.x only. Earlier iterations used old-school SysV init scripts and runlevels.

[jonvarteresian]

so dbus-daemon isnt actually dead here, just reaaaaaaaaally slow in response. messages still make it from lipc to the reader they just take minutes instead of seconds.

i attempted to watch the process with dbus-monitor --system but nothing of interest shows up in there to help id this problem.

kindle is still quite responsive as well.

EDIT. so i now believe this is due to blowing by the 1024 file descriptor limit in dbus. i can actually see the file descriptor list grow via lsof for the dbus-daemon. Now the question is why are these files being left open.

apparently you can change the limit from 1024 to 65535 but I have not been able to do so . ulimit -n lists it at 1024. anybody know how to change that?

[jonvarteresian]

ok, so i have a theory now.

everytime i load a pdf whether via the kindle directly, or via lipc commands and then close it, I am left with an additional file descriptor (FD).

when i open anything else, book, text doc, etc I am not left with any additional FDs

since the dbus-daemon starts with somewhere around 100 open file descriptors, as soon as i open my 920 pdfs, i run out of FDs for the dbus-daemon and it goes into a tight 100% CPU loop. dbus is limited to 1024 FDs. (This particular response for dbus is all over the web).

now to understand why the FD is being left open, or figure out a way to kill them since they are dead anyway.

[NiLuJe]

If you can reproduce it with vanilla behavior, that sounds like a bug, a good old resource leak. Which FW are you running?

[jonvarteresian]

Well, I can reproduce the problem and recover from it at will now.

I open a static file on my PW4 around 920 times and dbus-daemon pegs at 100%.
I look at

lsof |grep dbus-dae

and see 1024 open FDs
I delete a few FDs with gbd (p close(FD)) and everything starts right back up until i hit 1024 again

its pretty clear to me that the pdfreader is leaving open FDs. And from poking around the web this is a known issue with dbus when you bump over the 1024 limit. There are ways to increase this limit apparently, but not with amazon's version of this build that I can see. They want you to edit the /etc/security/limits.conf but it doesnt exist in the FS.

I tried via ulimit as well

ulimit -S -n 4096

for example but it had no affect on operation.

I then took my pdf and converted it to a mobi and azw3 and can display both of those thousands of times with my same code with no open FDs or problems. So for my immediate need i need to figure out how to convert my PDF to azw3 via python and move forward.

my kindle is running 5.10.0.2 and is a PW4 (G000 PP).

thanks for all the help on this one.

[j.p.s]

How does your PDF get generated and from what sort of sources?

[jonvarteresian]

Hi,

Using PIL (python) mainly, however I tried various other sources as well with the same results. I tried microsoft print to pdf, and I tried a scan to pdf from my scanner.

All displayed correctly on the kindle, but all left an open FD at the end.

[j.p.s]

I am suggesting that you consider making azw3 without PDF as an intermediate step.

[jonvarteresian]

hi,

i have tried multiple sources of documents, with many generation methods, all work fine expect PDFs, which leave the FD open.

I wrote some code to go in via gdb and close the FD after I was done with it. My test which used to fail at 900-ish now has over 7000 cycles, so I am calling this done.

thanks for the help