[Device Plugin] iOS reader applications

[HowardBrazee]

Originally Posted by theducks

You are 14 versions behind 2.26
But that may not be your only issue.

The latest Apple updates are causing problems

I upgraded. Now when I start Calibre, I get the initialization screen, and have to force kill Calibre after a while.

[Wyndham]

I just wanted to express my appreciation for all the effort being put into calibre and its various plugins.

Thank you, thank you, thank you

[ChaosForged]

Still working on this.

I just finished porting a bunch of changes from the main branch into the windows port, and will be testing again tonight.

Unfortunately its still enough of a black box to me that I'm forced to try things out in chunks, but I think I'm in the right area now at least.

[Philantrop]

Originally Posted by ChaosForged

I just finished porting a bunch of changes from the main branch into the windows port, and will be testing again tonight.

If you're talking about the libimobiledevice stack of libraries: That's not the right approach. You should take the proper parts of the Windows port and add them to the primary repository's build system. The maintainers explicitly stated they'll gladly accept such patches.

That has already been done for libplist. Another good source of inspiration is libusb.

Apart from that, AFAICS, the necessary changes for iOS 8.3 support simply haven't been implemented yet by the libimobiledevice guys so there's nothing to port really.

[ShellShock]

I agree with Philantrop. I have updated my Windows port of libimobiledevice from the latest master at https://github.com/libimobiledevice. This port is the source that is currently used in Calibre (Windows). Debugging the result against Marvin on IOS 8.3 gives this:

DEBUG:   85.8 libiMobileDevice:_house_arrest_send_command(command=u'VendContainer' appid='com.appstafarian.Marvin') 
17:09:13 [0x1ca8] .\src\property_list_service.c:127 internal_plist_send(): sending 305 bytes
17:09:13 [0x1ca8] .\src\service.c:140 service_send(): sending 4 bytes
17:09:13 [0x1ca8] ->usbmuxd_send(2628)
17:09:13 [0x1ca8] <-usbmuxd_send(2628)
17:09:13 [0x1ca8] .\src\service.c:140 service_send(): sending 305 bytes
17:09:13 [0x1ca8] ->usbmuxd_send(2628)
17:09:13 [0x1ca8] <-usbmuxd_send(2628)
17:09:13 [0x1ca8] .\src\property_list_service.c:132 internal_plist_send(): sent 305 bytes
17:09:13 [0x1ca8] .\src\property_list_service.c:133 internal_plist_send(): printing 305 bytes plist:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>Command</key>
	<string>VendContainer</string>
	<key>Identifier</key>
	<string>com.appstafarian.Marvin</string>
</dict>
</plist>
DEBUG:   85.8 libiMobileDevice:_house_arrest_get_result() 
17:09:13 [0x1ca8] ->usbmuxd_recv_timeout(2628, 4)
17:09:13 [0x1ca8] <-usbmuxd_recv_timeout(2628, 4)
17:09:13 [0x1ca8] .\src\property_list_service.c:191 internal_plist_receive_timeout(): initial read=4
17:09:13 [0x1ca8] .\src\property_list_service.c:200 internal_plist_receive_timeout(): 249 bytes following
17:09:13 [0x1ca8] ->usbmuxd_recv_timeout(2628, 249)
17:09:13 [0x1ca8] <-usbmuxd_recv_timeout(2628, 249)
17:09:13 [0x1ca8] .\src\property_list_service.c:213 internal_plist_receive_timeout(): received 249 bytes
17:09:13 [0x1ca8] .\src\property_list_service.c:239 internal_plist_receive_timeout(): printing 249 bytes plist:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>Error</key>
	<string>InstallationLookupFailed</string>
</dict>
</plist>
DEBUG:   85.8             ERROR: InstallationLookupFailed

The InstallationLookupFailed error is an open issue on the libimobiledevice GitHub https://github.com/libimobiledevice/ifuse/issues/24. If this is ever fixed we can add the fix into the Calibre source.

[ShellShock]

Originally Posted by Philantrop

If you're talking about the libimobiledevice stack of libraries: That's not the right approach. You should take the proper parts of the Windows port and add them to the primary repository's build system. The maintainers explicitly stated they'll gladly accept such patches.

There may be a problem with that: the primary repository uses mingw for the Windows port, but Calibre builds its C/C++ libraries from source with Visual Studio 2008; Kovid has already said he does not want to mix mingw binaries with VS binaries within Calibre because of incompatible C runtimes. Thus, we have to port the primary repository to VS 2008 so it can be built into Calibre; currently Calibre is using my libimobiledevice port here: https://github.com/Polyfun (which does not work with IOS 8.3). I have locally updated my port to include the latest source from the master, but it still does not work with IOS 8.3 because of the InstallationLookupFailed error.

[Philantrop]

Originally Posted by ShellShock

There may be a problem with that: the primary repository uses mingw for the Windows port

It shouldn't be a great technical challenge to support both mingw *and* MSVC.

And, as I said, they're happy to accept such patches:

Originally Posted by Martin Szulecki (primary libimobiledevice maintainer)

Contributions to add MSVC support are welcome to the project but
should integrate properly (other projects like libusb are a good
inspiration on how to do it).

So far only cross compilation with MinGW is the way we're supporting
and been compiling for Windows targets.

[ShellShock]

In calibre\src\calibre\devices\idevice\libimobiledevi ce.py, mount_ios_app, we have this:

self._house_arrest_send_command(command='VendContainer', appid=app_id)

I believe VendContainer access has been withdrawn by Apple in IOS 8.3. I have tried VendDocuments instead, and that still works, but I think this only gives access to the epubs in Marvin, and the IOSRA plugin can no longer access other files in Marvin that it needs, e.g., iOS reader applications\readers\Marvin_overlays.py, can_handle tries to access u'/Library/calibre/connected.xml', which errors with u'Permission denied (10)'.

So two options, neither of which may be feasible:

• Rewrite IOSRA (and other Marvin plugins?) to work with just VendDocuments access. But presumably they absolutely need the Marvin sqllite database, which requires VendContainer.
  
• Get the Marvin developer to put all the necessary files into the documents area in Marvin. Maybe unlikely.

This is looking less like a problem that can be fixed by changes to the libimobiledevice library, and more like changes are required to the plugins/Marvin.

[AndrewG]

ShellShock,

This GitHub thread (Cannot send VendDocuments command on HouseArrestClient #193) indicates that VendContainer appears to only work under iOS 8.3 for apps installed other than via the Apple App Store.

https://www.google.com.au/url?sa=t&r...92291466,d.cGU

[ShellShock]

Yes I saw that, and I have no idea if it is possible to install Marvin that way, or how to do it.

EDIT: I have checked out using AdHoc/XCode to install Marvin: this requires the target idevice uuids to be known to the (Marvin) developer so they can create a provisioning profile specific to those idevices. Not really practical for an app normally distributed through the app store, which is installed on arbitrary idevices where the developer does not know the uuids.

[Truong Mo]

calibre, version 1.47.0
ERROR: Install Plugin Failed: A problem occurred while installing this plugin. This plugin will now be uninstalled. Please post the error message in details below into the forum thread for this plugin and restart Calibre.

Traceback (most recent call last):
File "site-packages/calibre/gui2/dialogs/plugin_updater.py", line 713, in _install_clicked
File "site-packages/calibre/customize/ui.py", line 383, in add_plugin
File "site-packages/calibre/customize/ui.py", line 550, in initialize_plugin
InvalidPlugin: Initialization of plugin Traceback (most recent call last):
File "site-packages/calibre/customize/ui.py", line 544, in initialize_plugin
File "calibre_plugins.ios_reader_apps.__init__", line 916, in initialize
File "calibre_plugins.ios_reader_apps.__init__", line 1245, in _get_connected_device_info
InitialConnectionError: Unable to connect to iDevice
failed with traceback:
Traceback (most recent call last):
File "site-packages/calibre/customize/ui.py", line 544, in initialize_plugin
File "calibre_plugins.ios_reader_apps.__init__", line 916, in initialize
File "calibre_plugins.ios_reader_apps.__init__", line 1245, in _get_connected_device_info
InitialConnectionError: Unable to connect to iDevice

What's wrong?

[ShellShock]

What version of IOS are you using? Please note that IOS 8.3 is not supported by the plugin. I also note from https://ipsw.me/all that Apple have stopped signing IOS 8.2 so it is no longer possible to downgrade 8.3 to 8.2. So, our options are:

• If you are already on IOS 8.2 (or earlier), do not upgrade to IOS 8.3 and carry on using the IOSRA plugin (e.g., to connect to Marvin).
• Or, if you are on IOS 8.2 and want to upgrade to 8.3, accept that you will no longer be able to use the IOSRA plugin.
• Or, if you are already on IOS 8.3, then the IOSRA plugin will just not work.

This situation may change in the future if:

• Apple relent, and decide to open up file access again in a later version of IOS (seems unlikely, but who knows?)
• Or Marvin etc and the IOSRA plugin are rewritten to share files via the documents area, which is still open. Again seems unlikely--I am not aware of anyone working on this.
• Or a jailbreak becomes available for IOS 8.3 (or a later version). The plugin should still work with jailbroken idevices, but there is no jailbreak (yet) for IOS 8.3. The rumour is that there may never be one, and the jailbreakers are holding off to jailbreak 8.4 when it comes out instead. Obviously it is a personal decision as to whether to jailbreak or not.

So this could be the end of the road for the IOSRA plugin.

[Beamis]

Originally Posted by ShellShock

What version of IOS are you using? Please note that IOS 8.3 is not supported by the plugin. I also note from https://ipsw.me/all that Apple have stopped signing IOS 8.2 so it is no longer possible to downgrade 8.3 to 8.2. So, our options are:

• If you are already on IOS 8.2 (or earlier), do not upgrade to IOS 8.3 and carry on using the IOSRA plugin (e.g., to connect to Marvin).
• Or, if you are on IOS 8.2 and want to upgrade to 8.3, accept that you will no longer be able to use the IOSRA plugin.
• Or, if you are already on IOS 8.3, then the IOSRA plugin will just not work.

This situation may change in the future if:

• Apple relent, and decide to open up file access again in a later version of IOS (seems unlikely, but who knows?)
• Or Marvin etc and the IOSRA plugin are rewritten to share files via the documents area, which is still open. Again seems unlikely--I am not aware of anyone working on this.
• Or a jailbreak becomes available for IOS 8.3 (or a later version). The plugin should still work with jailbroken idevices, but there is no jailbreak (yet) for IOS 8.3. The rumour is that there may never be one, and the jailbreakers are holding off to jailbreak 8.4 when it comes out instead. Obviously it is a personal decision as to whether to jailbreak or not.

So this could be the end of the road for the IOSRA plugin.

And this is why I'm selling the iPad and buying an Android or Surface.

The Marvin developer hasn't done jack for a loooong time, in terms of enhancing the product. Marvin is an ok reader, but has long-standing, glaring holes (PDFs, page numbering). That, on top of Apple continuing to load up the device with closed-ecosystem security, enough's enough.

[eschwartz]

Originally Posted by Beamis

And this is why I'm selling the iPad and buying an Android or Surface.

If you get an Android, be sure to check out chaley's excellent Calibre Companion. It interfaces wirelessly with calibre and has been written expressly for managing a calibre library synced to Android.
Excellent support, since chaley still occasionally codes on calibre itself!

You would still need another app to open and read a book, but CC makes an excellent library organizer.

[truth1ness]

Darn it, I just completed moving all my ebooks into Calibre and was about to load up Marvin on my iPhone. So what are you guys doing to get books onto your idevice now? I just want to sync my top priority marked books to my iPhone. Just start manually dragging them to something like dropbox?