[Solved] Not so strange random crashing

[cubemike99]

I have a jailbroken paperwhite on 5.3.3. It has usbnetworking installed with ssh enabled. I am using backdoorlock to prevent updates. Every so often (from 10 minutes to a few hours since the last time) my kindle crashes and starts rebooting from the boy under tree screen with horizontal progress bar. After it gets to the home screen, usually within 30 seconds it reboots again the same way. After this episode, it is fine for another 10 minutes to a few hours until the same thing happens again. The last time it happened, I managed to sneak a peak at /var/log/messages. One line mentioned a crash, but that the crash report was being written to /dev/null. I activated debug logging with /usr/bin/debugOn.sh and am since waiting until the next crash to see if detailed info is available. Anybody have a similar experience?

P.S.: This crashing truly is random. It even happened a few times while the kindle was on its screensaver with me not logged in with ssh.

SOLVED: Everything seems to point to the indexer having problems with certain books. No real way to prevent it except to deal with a few possible reboots after uploading new books.

[cubemike99]

Now this is interesting. I was snooping around the commandline, and was shocked to find I was drawing 85 mA while on screensaver. I ran a ps aux an was shocked for a second time that one process, while in screensaver, was using 30% of both processor and memory. The attached pastie has the whole process. I see exception handlers in there, so I'm assuming this is not a good thing to be running.

[hawhill]

Well, when you're snooping around the commandline, you're not strictly just "on screensaver". Or rather, it's not the same as if you were not snooping around the command line. How did you measure the CPU utilization? Memory utilization does not make a difference regarding energy consumption. Where are you seeing exception handlers in there? I just see a bunch on jars in a loooooong classpath.

tl;dr: looks pretty normal.

[knc1]

Originally Posted by cubemike99

Now this is interesting. I was snooping around the commandline, and was shocked to find I was drawing 85 mA while on screensaver. I ran a ps aux an was shocked for a second time that one process, while in screensaver, was using 30% of both processor and memory. The attached pastie has the whole process. I see exception handlers in there, so I'm assuming this is not a good thing to be running.

1) A screensaver displayed != system suspend (as Hawhill noted).

2) Ah, no, you don't want to prevent running cvm. That is the "heart" of Amazon's Java system. (The "soul" of the system was left on the floor at lab126.)

3) What may be significant is the length of that command line - -
What maximum command line length the 5.3.3 Busybox is built with should be checked.
Busybox can be built with (as I recall) a 4096 character maximum length.
That command line is reported by the pastie to be 4097 (which over-wrote the trailing zero, if it is a 4096 character limit).
Note: Amazon is running a fairly old version of Busybox - it has been a long time since I built versions that old - I may be wrong about there being a command line limit in Ash.
Note 2: There may be a single line length limit in that pastie site. This all may be a red herring.

4) If the start-up of cvm is approaching the shell's command line limit, then a more significant (error?) is that lab126 is including all supported locale jars in the classpath.
Maybe that is required by their design, but if not, then it should be only loading the current locale jars. (Oops)

If you are feeling adventurous - - -
Install 5.3.4 - (remember to restore the things which need restoring, listed in the Jailbreak thread)
Check that cvm command length again (pasted somewhere like you just did).
Note: Or better, just attach as a compressed text file to your reporting post. No external site, line length limit in that case.

Repeat for 5.3.5 and 5.3.6, report on each.
The first post has a link to an off-site archive of the various Amazon image releases, these are "complete" images. You can "update" to one of the earlier firmware versions after testing all of them.

OR
The device is still under warranty - call Amazon CS and report it "broke" - play dumb about the details - get it replaced.

[NiLuJe]

As for the RAM/CPU usage, Java has a pretty specific memory handling, and, IIRC, on the PW, it sits nicely between 60 and 70MB, which is roughly 30% of the 250MB of RAM available, so that also looks okay.

And, by default, ps reports the average total CPU usage over the complete lifetime of a process (ie. it's not a snapshot on the cpu usage at the exact moment you ran ps), so, depending on how early after boot/framework boot you launch ps, since the framework pretty much sits at 100% CPU during boot, that might not be completely extraordinary either.

That said, the most common reason for the framework *really* looping at 100% cpu is an indexing loop. Check the logs to see if it's looping on a book (beside the usual compacting/cleanup tasks).

[cubemike99]

Thanks for all the info. As soon as I wanted the device to crash so I could look at the logs, it stopped crashing. I updated to 5.3.6 just because, and cvm's classpath is still ridiculously long, so it cuts off in the middle of a jar's pathname. The interesting question is whether this is a display issue or if the classpath really is being cut off. To that end, would it be possible to edit the command that invokes cvm to have only the relevant jars included. Come to think of it, why didn't amazon simply use "/opt/amazon/ebook/lib/*"?

[ixtab]

Originally Posted by cubemike99

Thanks for all the info. As soon as I wanted the device to crash so I could look at the logs, it stopped crashing. I updated to 5.3.6 just because, and cvm's classpath is still ridiculously long, so it cuts off in the middle of a jar's pathname. The interesting question is whether this is a display issue or if the classpath really is being cut off.

nope, it isn't cut off. If it was, you could be sure that cvm would crash all the time, for every single K5 user out there...

To that end, would it be possible to edit the command that invokes cvm to have only the relevant jars included. Come to think of it, why didn't amazon simply use "/opt/amazon/ebook/lib/*"?

All of the jars that are "mentioned" are relevant in one way or another.

... And, come to think of it: was does "*" mean? Should it be the shell which understands and expands "*", or should it be the program? What difference does it make?

(to answer those rhetorical questions: yes, it's the shell, and no, it makes no (functional) difference whatsoever, except that if the shell already does it, the program doesn't have to worry about wildcard expansion, resulting in less complexity. Seems logical to have the shell do it - after all, it's part of its job description)

[knc1]

Rhetorical: Why would the device need all possible locale jars in the classpath at the same time?
Scratches head a bit - -

1) To switch locales in the gui without a classpath reload.
2) The locale entry for each entry in the selection gui is stored one-per-locale-jar.

The above is pure speculation.

@ixtab: OK, Maybe it is "by design" or at least an artifact of their design. We have certainly seen lab126 do seemingly stranger things.

@cubemike99: Lets go with "ps" being internally limited to displaying a single "memory page" worth of command line, rather than the whole thing.

Translation: I made a bad "first guess" as to the cause of the random restarts.
Question is still open.

[cubemike99]

So after a brief period with no random restarts post update to 5.3.6, they're back. If the logs are to be believed, it's segmentation faults that are making it restart. It's so bizarre, too. I just had it restart on me 4 times with no time in between.

[knc1]

Originally Posted by cubemike99

So after a brief period with no random restarts post update to 5.3.6, they're back. If the logs are to be believed, it's segmentation faults that are making it restart. It's so bizarre, too. I just had it restart on me 4 times with no time in between.

What is it that is seg-faulting?
Always the same thing?

hmm...
Maybe better, compress the log file and attach to a post so we can see the full context.

[knc1]

In addition to the above information - -
All details of your connection to the Kindle.
Kindle end, Host end, software both ends.

- - - -

I may have just been able to duplicate your problem, but I need lots and lots of information from your end.

[cubemike99]

The thing with /var/log/messages is it seems to wipe itself on each reboot, so it has to reboot while I'm looking at it for me to catch it. Additionally, messages only reported that there was a segfault and wrote the crash log to /dev/null even after debug logging was enabled. But, running showlog -f did give me the pid of the segfaulting process, but since the device and all its processes already restarted, the pid didn't give me much.

Is there a way to keep /var/log/messages persistent?

[knc1]

Originally Posted by cubemike99

Is there a way to keep /var/log/messages persistent?

Each re-boot should be triggering a run of logrotate, which would be saving the prior log file(s).

Ah, give me until tomorrow to do a bit of exploring on mine to test that and discover filenames.

Until then,
What connection to the Kindle are you using? (Wifi, USBnet, serial console, network console, etc)

What is your host machine and OS?

I really need that information to attempt to duplicate your problem.
The greater number of exact details you can give me about your entire setup, the better.

[cubemike99]

Alright here goes:

I usually connect using wifi enabled usbnet, and I keep usbnet on.
I'm running OSX 10.8 on a 2009 13 inch macbook pro and connect with the default terminal and ssh keys.
On the pw I have installed KUAL, kterm, and the temporary margin fix that was posted in the JBPatch thread. I'm still running 5.3.6. I also have extend usually mounted (though the pw restarts with and without it mounted). That's about all I can think of right now.

[knc1]

Originally Posted by cubemike99

Alright here goes:

I usually connect using wifi enabled usbnet, and I keep usbnet on.

Are you connecting over Wifi or are you connecting over USBnet?

Is the screensaver ever showing when it crashes?

Simplify things a bit -
Disable Wifi (airplane mode) and connect using Telnet over the USBnet.

See if that makes any difference to the pattern of crashing.

PS:
Version of:
USBnetworking
kTerm
KUAL
that are installed please.