Compiling Calibre?

[hobleyd]

Hello,

I am trying to compile Calibre for OSX (initially). I have followed the git repo at:

https://github.com/kovidgoyal/build-calibre

./osx builds the dependencies as expected. Albeit with a couple of changes to sources.json:

diff --git a/scripts/sources.json b/scripts/sources.json
index bd9c8e8..a345f8a 100644
--- a/scripts/sources.json
+++ b/scripts/sources.json
@@ -2,8 +2,8 @@
{
"name": "zlib",
"unix": {
- "filename": "zlib-1.2.8.tar.xz",
- "hash": "md5:28f1205d8dd2001f26fec1e8c2cebe37",
+ "filename": "zlib-1.2.10.tar.xz",
+ "hash": "sha256:9612bf086047078ce3a1c154fc9052113fc1a2a972 34a059da17a6299bd4dd32",
"urls": ["http://zlib.net/{filename}"]
}
},
@@ -401,13 +401,13 @@
"name": "qt",
"unix": {
"filename": "qt-everywhere-opensource-src-5.6.1-1.tar.xz",
- "hash": "sha1:b014552b636c5c3ad72d7f9f7b687fde179cf861 ",
- "urls": ["http://download.qt.io/official_releases/qt/5.6/5.6.1/single/{filename}"]
+ "hash": "md5:f44fd4fcae71087f90d9c68e9d8be449",
+ "urls": ["http://download.qt.io/official_releases/qt/5.6/5.6.1-1/single/{filename}"]
},
"windows": {
"filename": "qt-everywhere-opensource-src-5.6.1-1.zip",
- "hash": "sha1:72a7e4094a6d18691b8decdd2ecdb205a16cca6e ",
- "urls": ["http://download.qt.io/official_releases/qt/5.6/5.6.1/single/{filename}"]
+ "hash": "md5:9d7ea0cadcec7b5a63e8e83686756978",
+ "urls": ["http://download.qt.io/official_releases/qt/5.6/5.6.1-1/single/{filename}"]
}
},

but when I then try to build calibre with:

./osx calibre

I get as far as linking unrar and then it fails with the following message:

Linking unrar


g++ /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/strfn.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/filestr.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/arcread.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/rar.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/encname.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/isnt.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/sha1.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/filefn.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/timefn.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/smallfn.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/crc.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/match.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/consio.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/scantree.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/global.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/rarvm.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/rijndael.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/extract.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/unicode.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/crypt.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/system.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/archive.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/savepos.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/volume.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/options.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/list.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/unrar.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/file.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/pathfn.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/cmddata.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/errhnd.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/extinfo.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/find.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/rdwrfn.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/filcreat.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/getbits.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/rawread.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/ulinks.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/strlist.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/secpassword.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/unpack.o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/build-q6DtAH/objects/unrar/resource.o -o /var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/plugins-bEiwQe/unrar.so -Wall -headerpad_max_install_names -L/sw/sw/lib -bundle -undefined dynamic_lookup


* build took 55.0 seconds
Failed to find icns format icons
Connection to osx-calibre-build closed.

Mirroring data from VM...
Syncing osx-calibre-build:/sw
Syncing osx-calibre-build:/sources

Anyone any ideas?

Cheers,
David

[kovidgoyal]

unrar is linking, the problem is the missingicns icons

* build took 55.0 seconds

Failed to find icns format icons

That can be fixed by running

icons/icns/make_iconsets.py

on the host machine

[kovidgoyal]

https://github.com/kovidgoyal/build-...4e41c70fc890ae

[hobleyd]

Thanks for that; I updated the build-calibre repo and can confirm that the icons are now built automatically.

The next failure I get is:

Adding Python framework

Adding site-packages
/sw/sw/python/Python.framework/Versions/2.7/lib/python2.7/site-packages
/calibre/src
Traceback (most recent call last):
File "/scripts/main.py", line 59, in <module>
main(args)
File "/scripts/pkgs/build_calibre.py", line 55, in main
freeze(args, ext_dir, test_runner)
File "/scripts/freeze/osx/__init__.py", line 703, in main
Freeze(build_dir, ext_dir, test_runner, dont_strip=args.dont_strip, sign_installers=args.sign_installers)
File "/scripts/freeze/osx/__init__.py", line 171, in __init__
self.run(test_launchers)
File "/scripts/freeze/osx/__init__.py", line 183, in run
self.add_site_packages()
File "/scripts/freeze/osx/__init__.py", line 147, in ff
ret = func(*args, **kwargs)
File "/scripts/freeze/osx/__init__.py", line 496, in add_site_packages
shutil.rmtree(os.path.join(self.site_packages, 'calibre', 'plugins'))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 239, in rmtree
onerror(os.listdir, path, sys.exc_info())
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 237, in rmtree
names = os.listdir(path)
OSError: [Errno 2] No such file or directory: '/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-29xGdT/calibre.app/Contents/Resources/Python/site-packages/calibre/plugins'
Shared connection to osx-calibre-build closed.

What do I need to do to fix this?

Cheers,
David

[kovidgoyal]

https://github.com/kovidgoyal/build-...5cebd5473a75d4

[hobleyd]

Thanks Kovid; all your help has been invaluable. I am closer; the last problem I think is now with the automated tests at the end. I get:


================================================== ====================
ERROR: test_lzma (calibre.test_build.BuildTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-l6GVQW/calibre.app/Contents/Resources/Python/site-packages/calibre/test_build.py", line 49, in test_lzma
File "/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-l6GVQW/calibre.app/Contents/Resources/Python/site-packages/lzma/xz.py", line 429, in test_lzma2
File "/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-l6GVQW/calibre.app/Contents/Resources/Python/site-packages/calibre/utils/resources.py", line 73, in get_path
IOError: [Errno 2] No such file or directory: u'/private/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-l6GVQW/calibre.app/Contents/Resources/resources/template-functions.json'

================================================== ====================
ERROR: test_spell (calibre.test_build.BuildTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-l6GVQW/calibre.app/Contents/Resources/Python/site-packages/calibre/test_build.py", line 59, in test_spell
File "/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-l6GVQW/calibre.app/Contents/Resources/Python/site-packages/calibre/spell/dictionary.py", line 87, in <module>
File "/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-l6GVQW/calibre.app/Contents/Resources/Python/site-packages/calibre/spell/__init__.py", line 30, in parse_lang_code
File "/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-l6GVQW/calibre.app/Contents/Resources/Python/site-packages/calibre/utils/localization.py", line 368, in canonicalize_lang
File "/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-l6GVQW/calibre.app/Contents/Resources/Python/site-packages/calibre/utils/localization.py", line 319, in _load_iso639
IOError: [Errno 2] No such file or directory: '/private/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-l6GVQW/calibre.app/Contents/Resources/resources/localization/iso639.pickle'

================================================== ====================
ERROR: test_break_iterator (calibre.utils.icu_test.TestICU)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-l6GVQW/calibre.app/Contents/Resources/Python/site-packages/calibre/utils/icu_test.py", line 160, in test_break_iterator
File "/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-l6GVQW/calibre.app/Contents/Resources/Python/site-packages/calibre/spell/break_iterator.py", line 22, in split_into_words
File "/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-l6GVQW/calibre.app/Contents/Resources/Python/site-packages/calibre/utils/localization.py", line 420, in lang_as_iso639_1
File "/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-l6GVQW/calibre.app/Contents/Resources/Python/site-packages/calibre/utils/localization.py", line 368, in canonicalize_lang
File "/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-l6GVQW/calibre.app/Contents/Resources/Python/site-packages/calibre/utils/localization.py", line 319, in _load_iso639
IOError: [Errno 2] No such file or directory: '/private/var/folders/wx/3mz4l2mn14d707l5tmtsfnlh0000gp/T/t/frozen-l6GVQW/calibre.app/Contents/Resources/resources/localization/iso639.pickle'

----------------------------------------------------------------------
Ran 55 tests in 11.903s

FAILED (errors=3, skipped=6)

What have I missed now?

[kovidgoyal]

You forgot to bootstrap the python source checkout. You need to run

python setup.py bootstrap

on the host computer.

[hobleyd]

Ah; that was it - I have a compiled dmg now; thanks!