Unable to build calibre-6.29.0 on Ubuntu 23.04+

[xtradeb]

Hi!

I'm unable to build Calibre 6.29.0 on Ubuntu 23.10. I'm getting the following error:

SIPing 4 files...
/usr/bin/python3.11 -c from sipbuild.tools.build import main; main(); --verbose --no-make --qmake /usr/bin/qmake6
usage: -c [-h] [-V] [--quiet] [--verbose] [--build-dir DIR]
[--scripts-dir DIR] [--target-dir DIR] [--api-dir DIR]
[--no-compile] [--no-version-info] [--concatenate N] [--debug]
[--disabled-feature TAG] [--no-docstrings] [--pep484-pyi]
[--protected-is-public] [--no-protected-is-public] [--tracing]
-c: error: unrecognized arguments: --no-make --qmake /usr/bin/qmake6

Not sure if this helps, but the build is working on Ubuntu 22.04 (python3-pyqt6.sip/jammy 13.2.1-2build1), but is failing, at least, since Ubuntu 23.04 (python3-pyqt6.sip/lunar 13.4.1-1build1).

Can you please advise?

For reference, this is the recipe I'm following (with minor tweaks): https://github.com/debian-calibre/ca.../master/debian

Thank you!
Jhonny Oliveira
https://xtradeb.net

[kovidgoyal]

that means the version of sip-build on your system no longer supports some command line options calibre uses when building pyqt extensions. It should be a fairly simple patch to setup/build.py to fix it.

[kovidgoyal]

No actually those options come from pyqtbuild not sipbuild and they are present at least in pyqt-builder 1.15.3 No idea why they are missing on your system

[xtradeb]

I tried removing the offending parameters

- '--verbose', '--no-make', '--qmake', QMAKE
+ '--verbose'

and I get

SIPing 4 files...
/usr/bin/python3.11 -c from sipbuild.tools.build import main; main(); --verbose
-c: pyproject.toml: Unclosed array (at line 23, column 218)

I'm not a developer. It's best I wait for a more knowledgeable mind to crack this one.

[xtradeb]

For furhter reference:
- Ubuntu 22.04
- python3-pyqtbuild 1.12.2+dfsg-1
- python3-sipbuild 6.6.2+dfsg-1~xtradeb1
- python3-pyqt6.sip 13.4.0-1~xtradeb1

- Ubuntu 23.10
- python3-pyqtbuild 1.15.2+dfsg-1
- python3-sipbuild 6.7.11+dfsg-1
- python3-pyqt6.sip 13.5.2-1

Is there anything else I can provide or do to help sorting this out?

[kovidgoyal]

No idea, basically that error means sip-build is either not loading pyqt-build or pyqt-build is broken on the system. There isnt anything I can do about it from calibre. You need working pyqt-build to build calibre.

[xtradeb]

Hi!

I looked into the changes prior to the failure and I found a workaround. See patch below.
Both, my local environment and launchpad.net (https://launchpad.net/~xtradeb/+arch...-archive-extra) are affected by the lines I'm removing as part of the workaround.
Strangely enough, the official source build system is not: https://launchpad.net/ubuntu/+source/calibre/7.0.0+ds-1

I'm not very knowledgeable in python. So, could you you please provide me a small patch to dump those variables (QMAKE_*) into to the build log? Hopefully, we will be able to understand why the build fails from it.

--- a/setup/build.py
+++ b/setup/build.py
@@ -687,16 +687,6 @@
{abi_version}
sip-include-dirs = ["/usr/lib/python3/dist-packages/PyQt6/bindings"]

-[tool.sip.builder]
-qmake-settings = [
- 'QMAKE_CC = {self.env.cc}',
- 'QMAKE_CXX = {self.env.cxx}',
- 'QMAKE_LINK = {self.env.linker or self.env.cxx}',
- 'QMAKE_CFLAGS += {shlex.join(self.env.base_cflags)}',
- 'QMAKE_CXXFLAGS += {shlex.join(self.env.base_cxxflags)}',
- 'QMAKE_LFLAGS += {shlex.join(self.env.base_ldflags)}',
-]
-
[tool.sip.bindings.{ext.name}]
headers = {ext.headers}
sources = {sorted(ext.sources)}

Thanks!

[JSWolf]

Now you can try building calibre 7.10.

[kovidgoyal]

print(open(os.path.join(src_dir, 'pyproject.toml')).read())

[xtradeb]

Hi!

You can find resulting [tool.sip.builder] below, as you can see both QMAKE_*FLAGS are broken due to un-escaped quotes.

[tool.sip.builder]
qmake-settings = [
'QMAKE_CC = gcc',
'QMAKE_CXX = g++',
'QMAKE_LINK = g++',
'QMAKE_CFLAGS += -g -O2 -ffile-prefix-map=/build/calibre-60FUQS/calibre-7.1.0+ds=. -flto=auto -ffat-lto-objects -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection '-fdebug-prefix-map=/build/calibre-60FUQS/calibre-7.1.0+ds=/usr/src/calibre-7.1.0+ds-1~xtradeb1' -Wdate-time -D_FORTIFY_SOURCE=2',
'QMAKE_CXXFLAGS += -g -O2 -ffile-prefix-map=/build/calibre-60FUQS/calibre-7.1.0+ds=. -flto=auto -ffat-lto-objects -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection '-fdebug-prefix-map=/build/calibre-60FUQS/calibre-7.1.0+ds=/usr/src/calibre-7.1.0+ds-1~xtradeb1' -Wdate-time -D_FORTIFY_SOURCE=2',
'QMAKE_LFLAGS += -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -Wl,-z,relro -Wl,-z,now',
]

Any ideas?

Thank you!

[xtradeb]

Hi again!

I came up with this patch:

--- a/setup/build.py
+++ b/setup/build.py
@@ -692,8 +692,8 @@
'QMAKE_CC = {self.env.cc}',
'QMAKE_CXX = {self.env.cxx}',
'QMAKE_LINK = {self.env.linker or self.env.cxx}',
- 'QMAKE_CFLAGS += {shlex.join(self.env.base_cflags)}',
- 'QMAKE_CXXFLAGS += {shlex.join(self.env.base_cxxflags)}',
+ 'QMAKE_CFLAGS += {shlex.join(self.env.base_cflags).replace("'",'"') }',
+ 'QMAKE_CXXFLAGS += {shlex.join(self.env.base_cxxflags).replace("'",'" ')}',
'QMAKE_LFLAGS += {shlex.join(self.env.base_ldflags)}',
]

It seems to work, but I'm not sure how reliable it might be. Any thoughts?

Thank you!

[kovidgoyal]

Well, the correct solution is to use the correct escaping algorithm for strings in toml. Sadly the python stdlib stupidly doesnt have a way to write toml despite python blessing toml as its build configuration format. I dont know what's going on in python land these days but they really do some of the most boneheaded things.

This should fix it as a practical matter
https://github.com/kovidgoyal/calibr...a3416b0bacfb8d

[xtradeb]

It works. Thank you very much!