[FileType Plugin] ACSM input plugin

[Leseratte_10]

If an error code is generated / returned by the server, it will be included in the log output. It won't be shown to the user (since a FileTypePlugin isn't allowed / able to show a popup message), but it can be seen in the log.

In this case, the plugin can't even send anything to the server as it fails to decrypt some of the activation information, so there's no error message for it to generate. In the next version it'll print a more useful error message (like "Authorization corrupt, try to re-authorize) into the log.

As for the adobe error codes, I've started a list of all of them I could find during development of this plugin: https://github.com/Leseratte10/acsm-...be-error-codes
Though, most of these are errors a normal end user will never encounter.

[Camelot]

Today I wanted to load an ASCM file, but the plugin was inactive, yesterday it worked. So I wanted to reinstall DeACSM 0.0.15, but it aborts with an error message.

calibre, version 5.42.0
ERROR: Unhandled exception: <b>InvalidPlugin</b>:Initialization of plugin Traceback (most recent call last):
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/_libcrypto_ctypes.py", line 304, in <module>
    libcrypto.EVP_PKEY_size.argtypes = [
  File "/usr/lib/python3.10/ctypes/__init__.py", line 387, in __getattr__
    func = self.__getitem__(name)
  File "/usr/lib/python3.10/ctypes/__init__.py", line 392, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /lib/x86_64-linux-gnu/libcrypto.so.3: undefined symbol: EVP_PKEY_size

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre_plugins.deacsm.__init__", line 184, in initialize
    from calibre_plugins.deacsm.libadobe import createDeviceKeyFile, update_account_path
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/usr/lib/calibre/calibre/customize/zipplugin.py", line 191, in exec_module
    exec(compiled, module.__dict__)
  File "calibre_plugins.deacsm.libadobe", line 29, in <module>
    from oscrypto import keys
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/keys.py", line 5, in <module>
    from ._asymmetric import parse_certificate, parse_private, parse_public
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_asymmetric.py", line 27, in <module>
    from .kdf import pbkdf1, pbkdf2, pkcs12_kdf
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/kdf.py", line 9, in <module>
    from .util import rand_bytes
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/util.py", line 14, in <module>
    from ._openssl.util import rand_bytes
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/util.py", line 6, in <module>
    from ._libcrypto import libcrypto, libcrypto_version_info, handle_openssl_error
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/_libcrypto.py", line 15, in <module>
    from ._libcrypto_ctypes import (
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/_libcrypto_ctypes.py", line 688, in <module>
    raise FFIEngineError('Error initializing ctypes')
oscrypto._ffi.FFIEngineError: Error initializing ctypes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/calibre/calibre/customize/ui.py", line 673, in initialize_plugin
    p.initialize()
  File "calibre_plugins.deacsm.__init__", line 187, in initialize
    from libadobe import createDeviceKeyFile, update_account_path
ModuleNotFoundError: No module named 'libadobe'
 failed with traceback:
Traceback (most recent call last):
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/_libcrypto_ctypes.py", line 304, in <module>
    libcrypto.EVP_PKEY_size.argtypes = [
  File "/usr/lib/python3.10/ctypes/__init__.py", line 387, in __getattr__
    func = self.__getitem__(name)
  File "/usr/lib/python3.10/ctypes/__init__.py", line 392, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /lib/x86_64-linux-gnu/libcrypto.so.3: undefined symbol: EVP_PKEY_size

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre_plugins.deacsm.__init__", line 184, in initialize
    from calibre_plugins.deacsm.libadobe import createDeviceKeyFile, update_account_path
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/usr/lib/calibre/calibre/customize/zipplugin.py", line 191, in exec_module
    exec(compiled, module.__dict__)
  File "calibre_plugins.deacsm.libadobe", line 29, in <module>
    from oscrypto import keys
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/keys.py", line 5, in <module>
    from ._asymmetric import parse_certificate, parse_private, parse_public
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_asymmetric.py", line 27, in <module>
    from .kdf import pbkdf1, pbkdf2, pkcs12_kdf
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/kdf.py", line 9, in <module>
    from .util import rand_bytes
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/util.py", line 14, in <module>
    from ._openssl.util import rand_bytes
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/util.py", line 6, in <module>
    from ._libcrypto import libcrypto, libcrypto_version_info, handle_openssl_error
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/_libcrypto.py", line 15, in <module>
    from ._libcrypto_ctypes import (
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/_libcrypto_ctypes.py", line 688, in <module>
    raise FFIEngineError('Error initializing ctypes')
oscrypto._ffi.FFIEngineError: Error initializing ctypes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/calibre/calibre/customize/ui.py", line 673, in initialize_plugin
    p.initialize()
  File "calibre_plugins.deacsm.__init__", line 187, in initialize
    from libadobe import createDeviceKeyFile, update_account_path
ModuleNotFoundError: No module named 'libadobe'


calibre 5.42  embedded-python: False is64bit: True
Linux-5.17.7-2-siduction-amd64-x86_64-with-glibc2.33 Linux ('64bit', 'ELF')
('Linux', '5.17.7-2-siduction-amd64', '#1 SMP PREEMPT siduction 5.17-7.1 (2022-05-13)')
Python 3.10.4
Interface language: None
Successfully initialized third party plugins: DeDRM (7, 0, 0) && EpubMerge (2, 13, 0) && Extract ISBN (1, 5, 0) && Open With (1, 7, 0) && Search The Internet (1, 8, 1)
Traceback (most recent call last):
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/_libcrypto_ctypes.py", line 304, in <module>
    libcrypto.EVP_PKEY_size.argtypes = [
  File "/usr/lib/python3.10/ctypes/__init__.py", line 387, in __getattr__
    func = self.__getitem__(name)
  File "/usr/lib/python3.10/ctypes/__init__.py", line 392, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /lib/x86_64-linux-gnu/libcrypto.so.3: undefined symbol: EVP_PKEY_size

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre_plugins.deacsm.__init__", line 184, in initialize
    from calibre_plugins.deacsm.libadobe import createDeviceKeyFile, update_account_path
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/usr/lib/calibre/calibre/customize/zipplugin.py", line 191, in exec_module
    exec(compiled, module.__dict__)
  File "calibre_plugins.deacsm.libadobe", line 29, in <module>
    from oscrypto import keys
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/keys.py", line 5, in <module>
    from ._asymmetric import parse_certificate, parse_private, parse_public
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_asymmetric.py", line 27, in <module>
    from .kdf import pbkdf1, pbkdf2, pkcs12_kdf
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/kdf.py", line 9, in <module>
    from .util import rand_bytes
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/util.py", line 14, in <module>
    from ._openssl.util import rand_bytes
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/util.py", line 6, in <module>
    from ._libcrypto import libcrypto, libcrypto_version_info, handle_openssl_error
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/_libcrypto.py", line 15, in <module>
    from ._libcrypto_ctypes import (
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/_libcrypto_ctypes.py", line 688, in <module>
    raise FFIEngineError('Error initializing ctypes')
oscrypto._ffi.FFIEngineError: Error initializing ctypes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/calibre/calibre/customize/ui.py", line 673, in initialize_plugin
    p.initialize()
  File "calibre_plugins.deacsm.__init__", line 187, in initialize
    from libadobe import createDeviceKeyFile, update_account_path
ModuleNotFoundError: No module named 'libadobe'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/calibre/calibre/gui2/preferences/plugins.py", line 326, in add_plugin
    plugin = add_plugin(path)
  File "/usr/lib/calibre/calibre/customize/ui.py", line 483, in add_plugin
    plugin = initialize_plugin(plugin, path_to_zip_file, PluginInstallationType.EXTERNAL)
  File "/usr/lib/calibre/calibre/customize/ui.py", line 678, in initialize_plugin
    raise InvalidPlugin((_('Initialization of plugin %s failed with traceback:')
calibre.customize.InvalidPlugin: Initialization of plugin Traceback (most recent call last):
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/_libcrypto_ctypes.py", line 304, in <module>
    libcrypto.EVP_PKEY_size.argtypes = [
  File "/usr/lib/python3.10/ctypes/__init__.py", line 387, in __getattr__
    func = self.__getitem__(name)
  File "/usr/lib/python3.10/ctypes/__init__.py", line 392, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /lib/x86_64-linux-gnu/libcrypto.so.3: undefined symbol: EVP_PKEY_size

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre_plugins.deacsm.__init__", line 184, in initialize
    from calibre_plugins.deacsm.libadobe import createDeviceKeyFile, update_account_path
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/usr/lib/calibre/calibre/customize/zipplugin.py", line 191, in exec_module
    exec(compiled, module.__dict__)
  File "calibre_plugins.deacsm.libadobe", line 29, in <module>
    from oscrypto import keys
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/keys.py", line 5, in <module>
    from ._asymmetric import parse_certificate, parse_private, parse_public
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_asymmetric.py", line 27, in <module>
    from .kdf import pbkdf1, pbkdf2, pkcs12_kdf
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/kdf.py", line 9, in <module>
    from .util import rand_bytes
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/util.py", line 14, in <module>
    from ._openssl.util import rand_bytes
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/util.py", line 6, in <module>
    from ._libcrypto import libcrypto, libcrypto_version_info, handle_openssl_error
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/_libcrypto.py", line 15, in <module>
    from ._libcrypto_ctypes import (
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/_libcrypto_ctypes.py", line 688, in <module>
    raise FFIEngineError('Error initializing ctypes')
oscrypto._ffi.FFIEngineError: Error initializing ctypes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/calibre/calibre/customize/ui.py", line 673, in initialize_plugin
    p.initialize()
  File "calibre_plugins.deacsm.__init__", line 187, in initialize
    from libadobe import createDeviceKeyFile, update_account_path
ModuleNotFoundError: No module named 'libadobe'
 failed with traceback:
Traceback (most recent call last):
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/_libcrypto_ctypes.py", line 304, in <module>
    libcrypto.EVP_PKEY_size.argtypes = [
  File "/usr/lib/python3.10/ctypes/__init__.py", line 387, in __getattr__
    func = self.__getitem__(name)
  File "/usr/lib/python3.10/ctypes/__init__.py", line 392, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /lib/x86_64-linux-gnu/libcrypto.so.3: undefined symbol: EVP_PKEY_size

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre_plugins.deacsm.__init__", line 184, in initialize
    from calibre_plugins.deacsm.libadobe import createDeviceKeyFile, update_account_path
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/usr/lib/calibre/calibre/customize/zipplugin.py", line 191, in exec_module
    exec(compiled, module.__dict__)
  File "calibre_plugins.deacsm.libadobe", line 29, in <module>
    from oscrypto import keys
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/keys.py", line 5, in <module>
    from ._asymmetric import parse_certificate, parse_private, parse_public
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_asymmetric.py", line 27, in <module>
    from .kdf import pbkdf1, pbkdf2, pkcs12_kdf
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/kdf.py", line 9, in <module>
    from .util import rand_bytes
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/util.py", line 14, in <module>
    from ._openssl.util import rand_bytes
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/util.py", line 6, in <module>
    from ._libcrypto import libcrypto, libcrypto_version_info, handle_openssl_error
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/_libcrypto.py", line 15, in <module>
    from ._libcrypto_ctypes import (
  File "/home/myusername/.config/calibre/plugins/DeACSM/modules/oscrypto/oscrypto/_openssl/_libcrypto_ctypes.py", line 688, in <module>
    raise FFIEngineError('Error initializing ctypes')
oscrypto._ffi.FFIEngineError: Error initializing ctypes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/calibre/calibre/customize/ui.py", line 673, in initialize_plugin
    p.initialize()
  File "calibre_plugins.deacsm.__init__", line 187, in initialize
    from libadobe import createDeviceKeyFile, update_account_path
ModuleNotFoundError: No module named 'libadobe'

Tested with Calibre from Debian and calibre-ebook.com (both 5.42.0) under Siduction 64bit (Debian/sid) with new profile.
The plugin works under Windows without any problems.
I think the problem will be the new python or ssl libraries under Debian/sid?

And many thanks for the awsome DeACSM 0.0.15 plugin.

[Leseratte_10]

This looks a lot like it's the same issue that currently happens on Ubuntu 22.04. Ubuntu 22.04 is based on debian unstable, so if 22.04 has the new openssl then Debian unstable / Siduction is going to have that, too.

I have already fixed this bug in the related oscrypto library (see https://github.com/wbond/oscrypto/pull/61 ), but that change hasn't been merged by the maintainer yet.

I will try to make the plugin use a forked / modified version of oscrypto until that fix gets into a new release and then the plugin should work again with OpenSSL 3.

[Camelot]

I just saw that last night on unstable there was a big version jump on OpenSSL (from 1.1.1-1 to 3.0.3-2).

[JSWolf]

I take it there's no issue with this plugin under Windows?

[Camelot]

Originally Posted by JSWolf

I take it there's no issue with this plugin under Windows?

No problem on Windows 11 for me

[JSWolf]

Originally Posted by Camelot

No problem under Windows.

Thank you.

GO WINDOWS!

[Leseratte_10]

Originally Posted by Camelot

I just saw that last night on unstable there was a big version jump on OpenSSL (from 1.1.1-1 to 3.0.3-2).

That's exactly what I mean. Debian & Ubuntu changed from OpenSSL 1.x to OpenSSL 3, which contains a bunch of incompatible changes. That means all software that uses OpenSSL is going to need to be updated. For software that's bundled with Ubuntu & Debian or is included in their repos, that's probably already happened, but because I was waiting for my PR to be merged, this is not yet the case with my plugin.

As I haven't received a response from the oscrypto maintainer within the last week as to when my bugfix is going to be merged, I'm just going to make my plugin patch the oscrypto code during plugin installation, so I don't have to wait for the oscrypto update.

Windows or MacOS or older Linux distributions are not affected by this bug, at least not currently. I'm not sure if MacOS also uses OpenSSL and if so, when that'll be updated to version 3.

[JSWolf]

I can't see MACOS coming with OpenSSL since it would be out with Apple's control.

[Leseratte_10]

That's good, because it means MacOS is not going to be affected by this bug then :P I don't know that much about MacOS, so I didn't know if they used OpenSSL or not.

The test version is now ready, in the 2nd post of this thread you can now download an experimental build of the plugin that works on Ubuntu 22.04.

Note that this version might require *two* restarts of Calibre after the installation (instead of the usual one restart) until it works properly, due to how I'm messing with the oscrypto library to get it to work. Once my changes are merged and a new version of oscrypto is released, this will no longer be necessary.

[Camelot]

The DeACSM_0.0.15-beta2204 works great with Siduction, Debian/unstable.
without errors from console:

Notifying optional server https://ofs.contentreserve.com/ACSNotificationService.svc/Notify
Weird Fulfillment Notification response: 
DeACSM v0.0.15: Downloading book ...
DeACSM v0.0.15: Loading book from https://acs.cdn.overdrive.com/ACSStore1/(…).epub
Download took 517 ms (HTTP 200)
That's a ZIP file -> EPUB
DeACSM v0.0.15: File successfully fulfilled ...
DeACSM v0.0.15: Executing plugin DeDRM ...
DeDRM v7.0.0: Trying to decrypt lh3do74u.epub
DeDRM v7.0.0: Verifying zip archive integrity
DeDRM v7.0.0: lh3do74u.epub is a secure Adobe Adept ePub
DeDRM v7.0.0: Trying Encryption key default_key
DeDRM v7.0.0: Decrypted with key default_key after 0.2 seconds
DeDRM v7.0.0: Finished after 0.2 seconds
DeACSM v0.0.15: Plugin returned path '/tmp/calibre_5.42.0_tmp_w902eah0/bje1t60i.epub', updating.

Thank you Leseratte_10.

[enuddleyarbl]

Just a heads up, Fedora 36 (just released) has apparently switched over to OpenSSL3 as well:

https://fedoraproject.org/wiki/Changes/OpenSSL3.0

[qfwfq]

Originally Posted by Leseratte_10

Beta versions

Ubuntu 22.04 and Fedora 36 come with OpenSSL 3, and the latest release of this plugin (0.0.15) does not yet support this.

If you're on Ubuntu 22.04 or Fedora 36 you can try this beta version that should work.

It seems to work.
But you have to cancel any data from previous installations that you can find in
~/.config/calibre/plugins

[DNSB]

Originally Posted by Leseratte_10

That's good, because it means MacOS is not going to be affected by this bug then :P I don't know that much about MacOS, so I didn't know if they used OpenSSL or not.

The last time I looked at MacOS, they were using the LibreSSL fork of the OpenSSL project as maintained by the OpenBSD people.

[Huwaetzel]

Originally Posted by Camelot

The DeACSM_0.0.15-beta2204 works great with Siduction, Debian/unstable.
without errors from console:

Notifying optional server https://ofs.contentreserve.com/ACSNotificationService.svc/Notify
Weird Fulfillment Notification response: 
DeACSM v0.0.15: Downloading book ...
DeACSM v0.0.15: Loading book from https://acs.cdn.overdrive.com/ACSStore1/(…).epub
Download took 517 ms (HTTP 200)
That's a ZIP file -> EPUB
DeACSM v0.0.15: File successfully fulfilled ...
DeACSM v0.0.15: Executing plugin DeDRM ...
DeDRM v7.0.0: Trying to decrypt lh3do74u.epub
DeDRM v7.0.0: Verifying zip archive integrity
DeDRM v7.0.0: lh3do74u.epub is a secure Adobe Adept ePub
DeDRM v7.0.0: Trying Encryption key default_key
DeDRM v7.0.0: Decrypted with key default_key after 0.2 seconds
DeDRM v7.0.0: Finished after 0.2 seconds
DeACSM v0.0.15: Plugin returned path '/tmp/calibre_5.42.0_tmp_w902eah0/bje1t60i.epub', updating.

Thank you Leseratte_10.

Same for me on Kubuntu 22.04! Thanks for the quick fix