Help needed with send metadata to device callback

[ShellShock]

I am trying to understand what is going on in the logs posted here: https://www.mobileread.com/forums/sho...&postcount=102

Specifically, this pattern occurs whenever there is an "database disk image is malformed" error:

Job: 5 Send metadata to device started
DEBUG:  686.4 iOSReaderApp:sync_booklists() 
DeviceJob: 5 Send metadata to device done, calling callback
database disk image is malformed

Traceback (most recent call last):
  File "site-packages\calibre\gui2\device.py", line 87, in run
  File "site-packages\calibre\gui2\device.py", line 532, in _sync_booklists
  File "C:\Users\Andrew\AppData\Local\Temp\calibre_bpv2hl\fa0znv_ios_local_db\Marvin_overlays.py", line 1009, in sync_booklists
  File "C:\Users\Andrew\AppData\Local\Temp\calibre_bpv2hl\fa0znv_ios_local_db\Marvin_overlays.py", line 2109, in _profile_db
DatabaseError: database disk image is malformed

DeviceJob: 5 Send metadata to device callback returned

It seems as if there are two threads in contention here:

• Job: 5 Send metadata to device started
• DEBUG: 686.4 iOSReaderApp:sync_booklists()

From the device.py code I have not been able to tell what callback is being called. If there are two threads here, it could explain why the error is occurring.

Or am I reading this wrong?

Thanks

[chaley]

Speaking from some knowledge of device drivers but no knowledge of the iOS plugins:

It appears that the iOS reader plugin is acting as a device driver. If so, then all methods in that plugin are called on the device manager thread, which is not the GUI thread. Every method in the plugin must complete its task during a single call. Unless you are very careful, it must not use sub-threads to do anything.

The sync_booklists call is usually called as part of a set of job steps.

1) [Device thread] Connect to device
2) [Device thread] Initialize the device
3) [Device thread] Get booklists (metadata) from device
4) [GUI thread] match books from device to calibre library
5) [Device thread] Send metadata back to the device (sync_booklists)
6) [GUI thread] Update views

The profile_db method will be called on the device thread. You can't trust the printout order because of threading.

You must be absolutely sure that nothing in the user interface plugin (marvin XD) calls any method in the iOS reader applications plugin, because that could cause threading problems.

I have no idea if this helps.

[kovidgoyal]

Look at devices/interface.py

Every single method in the Device object is called in the device thread, with a few exceptions, which are documented (search for thread in that file).

The calibre user interface is in the GUI thread, device operations are in the device thread, so as not to block the UI. The vast majority of device driver code is all executed in the device thread and calibre automatically takes care of dispatching events to the UI thread as appropriate. The few exceptions are documented in interface.py