Best way to debug Calibre crash?

[DMcCunney]

I just encountered a problem I have not seen before.

I have a current version of 64 bit Calibre, installed under Windows 10 Pro. I have an HP 7 Plus Android tablet running Android 4.4. My eBook library on tablet is on an external 32GB microSD card, and I use FBReader fo Android to view books on the tablet.

I connect to the tablet using a USB cable, and the tablet connects via MTP. When I plug in the tablet, Calibre sees a device is connected, and begins the process of looking for eBooks. That takes a fair bit of time, since there are thousands of volumes on device, so I do other things while I wait for it to finish. Once it's done I see Calibre's book list with checks beside volumes on device, and I can transfer new books and perhaps remove books currently on device.

Today, Calibre crashed and burned when the device was plugged in and it was searching for volumes. I got a Windows dialog box stating Calibre.exe had stopped working. In Windows Event Viewer, I saw

Faulting application name: calibre.exe, version: 2.85.0.0, time stamp: 0x591532f6
Faulting module name: wpd.pyd, version: 0.0.0.0, time stamp: 0x591532e8
Exception code: 0xc0000005
Fault offset: 0x000000000000174d
Faulting process id: 0x1994
Faulting application start time: 0x01d2cc43bb3fee13
Faulting application path: C:\Program Files\Calibre2\calibre.exe
Faulting module path: C:\Program Files\Calibre2\plugins2\wpd.pyd
Report Id: b830ef6f-f929-4aab-8e15-36580d1f56bb
Faulting package full name: 
Faulting package-relative application ID:

in the application log.

I restarted Windows and the tablet. I ran a file systems check on the volume where the Calibre library lives on Windows and it came up clean.

I shut down the tablet and ejected the 32GB microSD card, plugged it into an adapter, and ran a file systems check on it from the desktop. (The card is formatted FAT32.) That came up clean.

I restarted everything and tried again, and Calibre once again crashed communicating with the device.

What's the best way to debug this? Is there a good way to determine why the crash is occurring?

Looking at other threads here, I see that removing the Calibre metadata file on device and rebuilding might be a fix. I can do that if required - it wouldn't be the first time.

Suggestions? Comments? (NB: "Use wireless to connect to the tablet instead of a USB cable" is a very last resort matter. This has been working fine till now, and I'm averse to redoing my setup unless nothing else will work.)

Thanks in advance.
______
Dennis

[kovidgoyal]

On windows there is no good way -- the minimum you would need to do is rebuild calibre from source yourself with debugging symbold turned on and either use windebug or the visual studio debugger or failing those two stick lots of printf() statements into the relevant code path.

Windows is unfortunately the only major platform that makes it really hard to debug crashes -- it is on my todo list to setup some facilities for debugging crashes on user's windows computers for calibre but its a fair bit of work.

Generally with MTP crashes it is typically the windows portable device susbsystem returning unexpected data to calibre. Simply rebooting the device and the computer might fix it.

[DMcCunney]

Originally Posted by kovidgoyal

On windows there is no good way -- the minimum you would need to do is rebuild calibre from source yourself with debugging symbold turned on and either use windebug or the visual studio debugger or failing those two stick lots of printf() statements into the relevant code path.

Figures...

Generally with MTP crashes it is typically the windows portable device susbsystem returning unexpected data to calibre. Simply rebooting the device and the computer might fix it.

As mentioned, I tried that, and the crash occurred again after I had rebooted both devices.

What next? Remove the Calibre metadata file from the mobile device and let Calibre attempt to rebuild it from scratch?
______
Dennis

[kovidgoyal]

The two remaining options are:

1) remove metadata file, although I doubt that will help

2) Reset the device

[DMcCunney]

Originally Posted by kovidgoyal

The two remaining options are:

1) remove metadata file, although I doubt that will help

Can try.

2) Reset the device

As in "Reset to factory default?"

Not without my arm twisted behind my back and a gun pointed at my head. That's more last resort than "switch to wifi for connectivity between tablet and host".

What I'm curious about is why this just started happening. I can't think of any changes on the tablet side that might have caused a hiccup, and it worked fine yesterday.
______
Dennis

[DMcCunney]

Originally Posted by kovidgoyal

The two remaining options are:

1) remove metadata file, although I doubt that will help

And I did so, and it did work.

I popped the microSD card from the tablet, put in to an adapted, archived a copy of the Calibre metadata file, then removed it.

I put the card back in the tablet, rebooted it, then connected the tablet to the PC and ran Calibre. Calibre saw the device was connected, and began to create the metadata file from scratch. (It took an hour or so. I did other things.)

Calibre eventually created the metadata file, populated with entries from volumes found on the card, and the Calibre volume list had the expected check marks on volumes in Calibre found on the device. I was able to add more volumes and disconnect. As a double-check, I reconnected later. Calibre scanned the device with no problems, and I was able to transfer more volumes to the device and disconnect.

I recall having had a problem or two previously where Calibre appeared to hand while communicating with the device, and needed to be terminated from Task Manager. It appears that left the Calibre metadata on device in an inconsistent state that gave it heartburn later. Is there a function that can be used to sanity check the Calibre metadata on device?

Meanwhile, things seem back to normal.
______
Dennis

[kovidgoyal]

Glad to hear it.

[DMcCunney]

Originally Posted by kovidgoyal

Glad to hear it.

I'm certainly happy.

Curious fact: the metadata file created after I removed it and let Calibre rebuild from scratch is about 30MB. The one I removed was 53MB.

I'd guess the prior file was corrupted, and that caused the crashes when Calibre tried to read and update it.
______
Dennis

[kovidgoyal]

The file is a JSON file, which calibre reads using the python standard library json module. I would be *very* surprised if corrupted JSON files caused an actual crash.

The size difference is because IIRC when reading metadata from files on the device, book thumbnails are not stored, while when sending books from calibre to the device, they are.