ebook-convert epub to pdf RuntimeError

[xiatian]

I'm using the lastest linux binary,

ebook-convert b.epub b.pdf
1% Converting input to HTML...
Note 'webEngineArgs' are overridden by QTWEBENGINE_CHROMIUM_FLAGS
QRhiGles2: Failed to create temporary context
QRhiGles2: Failed to create context
This plugin does not support createPlatformVulkanInstance
QVulkanInstance: Failed to initialize Vulkan
Unable to detect GPU vendor.
InputFormatPlugin: EPUB Input running
on /root/test/b.epub
Found HTML cover titlepage.xhtml
Parsing all content...
34% Running transforms on e-book...
Merging user specified metadata...
Detecting structure...
Flattening CSS and remapping font sizes...
Source base font size is 12.00000pt
Removing fake margins...
Cleaning up manifest...
Trimming unused files from manifest...
Creating PDF Output...
67% Running PDF Output plugin
Traceback (most recent call last):
File "runpy.py", line 203, in _run_module_as_main
File "runpy.py", line 88, in _run_code
File "site.py", line 52, in <module>
File "site.py", line 48, in main
File "calibre/ebooks/conversion/cli.py", line 427, in main
File "calibre/ebooks/conversion/plumber.py", line 1265, in run
File "calibre/ebooks/conversion/plugins/pdf_output.py", line 190, in convert
File "calibre/ebooks/conversion/plugins/pdf_output.py", line 239, in convert_text
File "<frozen importlib._bootstrap>", line 1371, in _find_and_load
File "<frozen importlib._bootstrap>", line 1342, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 938, in _load_unlocked
File "bypy-importer.py", line 279, in exec_module
File "calibre/ebooks/pdf/html_writer.py", line 15, in <module>
File "<frozen importlib._bootstrap>", line 1371, in _find_and_load
File "<frozen importlib._bootstrap>", line 1342, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 938, in _load_unlocked
File "bypy-importer.py", line 279, in exec_module
File "html5_parser/__init__.py", line 31, in <module>
RuntimeError: html5-parser and lxml are using different versions of libxml2. This happens commonly when using pip installed versions of lxml. Use pip install --no-binary lxml lxml instead. libxml2 versions: html5-parser: (2, 9, 14) != lxml: (2, 15, 3)

However, 8.x version works normally.

[kovidgoyal]

Dont see how thats possible if you are using the official binaries. Since they come with their own bundled libxml. Use LD_DEBUG=libs to figure out where the spurious libxml is coming from.

[xiatian]

LD_DEBUG=libs ebook-convert b.epub b.pdf 2>&1 | grep -i "libxml2" > /tmp/ld_debug.log
cat /tmp/ld_debug.log
190956: find library=libxml2.so.16 [0]; searching
190956: trying file=/root/calibre/lib/libxml2.so.16
190956: calling init: /root/calibre/lib/libxml2.so.16
190961: find library=libxml2.so.16 [0]; searching
190961: trying file=/root/calibre/lib/libxml2.so.16
190960: find library=libxml2.so.16 [0]; searching
190960: trying file=/root/calibre/lib/libxml2.so.16
190960: calling init: /root/calibre/lib/libxml2.so.16
190961: calling init: /root/calibre/lib/libxml2.so.16
190956: find library=libxml2.so.2 [0]; searching
190956: trying file=/root/calibre/lib/libxml2.so.2
190956: trying file=/lib/x86_64-linux-gnu/libxml2.so.2
190956: calling init: /lib/x86_64-linux-gnu/libxml2.so.2
RuntimeError: html5-parser and lxml are using different versions of libxml2. This happens commonly when using pip installed versions of lxml. Use pip install --no-binary lxml lxml instead. libxml2 versions: html5-parser: (2, 9, 14) != lxml: (2, 15, 3)
190956: calling fini: /lib/x86_64-linux-gnu/libxml2.so.2 [0]
190956: calling fini: /root/calibre/lib/libxml2.so.16 [0]

[kovidgoyal]

https://www.mobileread.com/forums/sh...d.php?t=370085

[xiatian]

Originally Posted by kovidgoyal

https://www.mobileread.com/forums/sh...d.php?t=370085

This is what AI helped me and I think there maybe a fix for the next release.

Thanks for the pointer. Traced this further and it doesn't look like
a plugin issue — here's what I found:

1. calibre-customize -l shows only built-in plugins, no third-party
plugins installed at all.

2. Scanned every .so file under /root/calibre for a compiled ELF
dependency on libxml2.so.2:

find /root/calibre -type f -name "*.so*" | while read f; do
readelf -d "$f" | grep -q "libxml2.so.2" && echo "$f"
done

-> no output. Also did a strings scan across every file in
/root/calibre for the literal string "libxml2.so.2" -> also
nothing found. So nothing is linking against it at the ELF level,
confirming what you said.

3. However, running this with calibre's own bundled python:

/root/calibre/calibre-debug -c "import ctypes.util; print(ctypes.util.find_library('xml2'))"

returns: libxml2.so.2 (the system one, not calibre's bundled
libxml2.so.16)

So it looks like html5-parser is locating libxml2 at runtime via
ctypes.util.find_library() / dlopen rather than a compiled ELF NEEDED
dependency — which is why it doesn't show up in readelf or strings.
find_library() resolves through the system's ldconfig cache, which
only knows about whatever libxml2 the OS has registered — in this
case the system's own libxml2.so.2. It then tries to dlopen that
exact filename; the dynamic linker checks calibre's own lib dir first
(as seen in the LD_DEBUG trace) but finds no file by that name there
(calibre ships libxml2.so.16 under the new SONAME), so it falls
through to the system's copy instead. lxml, by contrast, loads
libxml2.so.16 directly and correctly picks up calibre's bundled copy.
End result: two different libxml2 versions loaded in the same
process.

I also tested this on a second, completely unrelated Debian 12 server
with a different software stack, and got the exact same error. So
it's not tied to any specific package pulling in libxml2 — the common
factor is simply that it's a Debian 12 host, and Debian 12's repo
still ships libxml2 2.9.14 under the old SONAME (libxml2.so.2), while
calibre's own bundled libxml2 has moved to the new SONAME (.so.16).
Given how many common packages transitively depend on libxml2, most
Debian 12 installs probably have some libxml2.so.2 registered in
ldconfig one way or another, so any Debian 12 host running the
official calibre binary seems likely to hit this.

As for a possible fix, I can see two independent ways to close this
gap (either one alone would probably be enough):

1. On the html5-parser side: instead of relying on
ctypes.util.find_library('xml2') (which resolves through the
system's ldconfig cache and has no awareness of calibre's own
bundled lib directory), it could resolve the libxml2 path the same
way lxml does — e.g. reuse the same shared library lxml already
loaded, or at least restrict the search to calibre's own lib dir
before falling back to a system-wide lookup.

2. On the calibre packaging side: since the bundled libxml2 SONAME
changed from .so.2 to .so.16 in this build, shipping a compat
symlink alongside it (libxml2.so.2 -> libxml2.so.16, inside
calibre's own lib dir) would make html5-parser's existing
find_library()-based lookup resolve correctly without any code
change.

For now I've applied the workaround you suggested:

ln -s libxml2.so.16 /root/calibre/lib/libxml2.so.2

which resolves it fine. Just wanted to share the fuller trace in case
it's useful.