Calibre "isolated" install for ARM

[plasmid]

calibre-server (arch linux version, installed via pacman) password protected login doesn't work. The problem is related to a wrong python library version.

But Calibre could be also installed as "isolated install" with all necessary files inside the installation directory. Since right python library version would be inside, the password protected login works (tested).

My problem is that isolated istallation script only works for 386 and x64 machines. How can I install it on an ARM system (arch linux)? Thank you.

[eschwartz]

The isolated install uses the prebuilt binaries. Kovid only pre-builds binaries for i686/x86_64.
Building calibre from source works just fine!

The password-protected login issues are a result of ArchLinux bug #43382 -- the solution is to rebuild the package yourself, and edit the PKGBUILD to no longer strip out the heavily-modified libs that are bundled with calibre.

Thank you ArchLinux developers for your care and concern!
They don't want to try getting those modifications pushed upstream. They don't want to stop patching out calibre. They don't want to drop calibre to the AUR, so people will be forewarned.

Idealistic purity wins out over user experience...

[plasmid]

Thank you for your answer.

Can you please tell me how to rebuild the armv7 package?

I know I have to download the sources, but... ...where can I find them (ABS is only for 386 and x64)?

Is the PKGBUILD inside the sources? How do I have to edit it to get the right libs?

Thank you for your patience.

[eschwartz]

ArchLinuxARM mostly uses the same sources as the main Arch repos, so you just need to pull from ABS.
http://archlinuxarm.org/developers/contributing

Arch Linux ARM uses the upstream Arch Linux packages and AUR for nearly all of its packages. However, some packages require tweaks to build and run properly on ARM devices. The PKGBUILDs for those packages are modified from the upstream source and placed into our git repository on GitHub. They are kept up to date daily with the rest of the packages, and may move in or out of git depending on (in)compatible changes from the upstream release.

Assuming you use yaourt, just do:

yaourt -Sb calibre

Then, when prompted to edit the PKGBUILD via your favorite editor, delete or comment out lines 35-38.

While you are at it, you can also remove "python2-{six,cherrypy,html5lib,chardet}" from the depends array.

PKGBUILD:

# $Id: PKGBUILD 135580 2015-06-19 14:11:19Z jelle $
# Maintainer: Jelle van der Waa <jelle@vdwaa.nl>
# Maintainer: Daniel Wallace <danielwallace at gtmanfred dot com>
# Contributor: Giovanni Scafora <giovanni@archlinux.org>
# Contributor: Petrov Roman <nwhisper@gmail.com>
# Contributor: Andrea Fagiani <andfagiani _at_ gmail dot com>
# Contributor: Larry Hajali <larryhaja@gmail.com>

pkgname=calibre
pkgver=2.31.0
pkgrel=1
pkgdesc="Ebook management application"
arch=('i686' 'x86_64')
url="http://calibre-ebook.com/"
license=('GPL3')
depends=('python2-dateutil' 'python2-cssutils'
         'python2-mechanize' 'podofo' 'libwmf'
         'imagemagick' 'chmlib' 'python2-lxml' 'libusbx'
         'python2-pillow' 'shared-mime-info' 'python2-dnspython'
         'python2-pyqt5' 'python2-psutil' 'icu' 'libmtp' 'python2-dbus'
         'python2-netifaces' 'python2-cssselect' 'python2-apsw' 'qt5-webkit'
         'qt5-svg' 'python2-pygments' 'mtdev')
makedepends=('python2-pycountry' 'qt5-x11extras' )
optdepends=('ipython2: to use calibre-debug'
            'udisks: required for mounting certain devices'
)
install=calibre.install
source=("http://download.calibre-ebook.com/${pkgver}/calibre-${pkgver}.tar.xz")
md5sums=('75237f9e11a2d5eefde8dfc19d6de5a4')

prepare(){
  cd "${srcdir}/${pkgname}-${pkgver}"

  # Remove unneeded files and libs
  # NONONONO bad Arch, keep these...
  #rm -rf resources/${pkgname}-portable.* \
  #       src/cherrypy \
  #       src/html5lib \
  #       src/chardet

  sed -i "s/shlex.split(ldflags)/& + ['-fPIC']/" setup/extensions.py

  # Use python2
  sed -i 's:\(env[ ]\+python$\|/usr/bin/python$\):\12:g' $(find . -regex ".*\.py\|.*\.recipe")

  # Desktop integration (e.g. enforce arch defaults)
  sed -e "/self.create_uninstaller()/,/os.rmdir(config_dir)/d" \
      -e "/\(cc('xdg-icon-resource\|self.icon_resources.append\|'128'))\)/d" \
      -e "/render_img/ s/\('calibre-.*\.png'\)/os.path.join(dir, \1)/g" \
      -e "/dir, 'calibre-lrf.png'/i \
\                dir = os.path.join(self.opts.staging_sharedir,'../pixmaps')\n\
\                os.mkdir(dir)" \
      -e "/f = open/ s/\('calibre-.*\.desktop'\)/os.path.join(dir, \1)/g" \
      -e "/dir, 'calibre-lrfviewer.desktop'/i \
\                dir = os.path.join(self.opts.staging_sharedir,'../applications')\n\
\                os.mkdir(dir)" \
      -e "s/'ctc-posml'/'text' not in mt and 'pdf' not in mt and 'xhtml'/" \
      -e "s/^Name=calibre/Name=Calibre/g" \
      -i  src/calibre/linux.py
}

build() {
  cd "${srcdir}/${pkgname}-${pkgver}"

  LANG='en_US.UTF-8' python2 setup.py build
  # LANG='en_US.UTF-8' python2 setup.py resources

  # Don't build translations since building them is broken badly
  #LANG='en_US.UTF-8' python2 setup.py translations
}

package() {
  cd "${srcdir}/${pkgname}-${pkgver}"

  # Fix the environment module location
  sed -i -e "s|(prefix=.*)|(prefix='$pkgdir/usr')|g" setup/install.py

  install -d "${pkgdir}/usr/lib/python2.7/site-packages" \
             "${pkgdir}/usr/share/zsh/site-functions"

  LANG='en_US.UTF-8' python2 setup.py install --root="${pkgdir}" --prefix=/usr \
    --staging-bindir="${pkgdir}/usr/bin" \
    --staging-libdir="${pkgdir}/usr/lib" \
    --staging-sharedir="${pkgdir}/usr/share"

  # Compiling bytecode FS#33392
  python2 -m compileall "${pkgdir}/usr/lib/calibre/"
  python2 -O -m compileall "${pkgdir}/usr/lib/calibre/"
}

[plasmid]

Doing "yaourt -Sb calibre", this is my PKGBUILD:

# $Id: PKGBUILD 74107 2012-07-21 08:16:51Z giovanni $
# Maintainer: Giovanni Scafora <giovanni@archlinux.org>
# Contributor: Petrov Roman <nwhisper@gmail.com>
# Contributor: Andrea Fagiani <andfagiani _at_ gmail dot com>

pkgname=calibre
pkgver=0.8.61
pkgrel=1
pkgdesc="Ebook management application"
arch=('i686' 'x86_64')
url="http://calibre-ebook.com/"
license=('GPL3')
depends=('python2-dateutil' 'python2-cssutils' 'python2-cherrypy'
         'python-mechanize' 'podofo' 'libwmf' 'python2-beautifulsoup3'
         'imagemagick' 'poppler-qt' 'chmlib' 'python2-lxml' 'libusbx'
         'python-imaging' 'shared-mime-info' 'python2-dnspython'
         'libunrar' 'python2-pyqt' 'icu')
makedepends=('python2-pycountry')
optdepends=('ipython2: to use calibre-debug')
install=calibre.install
source=("http://calibre-ebook.googlecode.com/files/${pkgname}-${pkgver}.tar.xz"
        'desktop_integration.patch'
        'calibre-mount-helper')
md5sums=('fb1c71f1a52f4a9d5f7954416c095249'
         'c28876d95936f901cc0f361754458ab0'
         '675cd87d41342119827ef706055491e7')

build() {
  cd "${srcdir}/${pkgname}"

  #rm -rf src/{cherrypy,pyPdf}
  rm -rf src/cherrypy
  sed -i -e "s/ldflags = shlex.split(ldflags)/ldflags = shlex.split(ldflags) + ['-fPIC']/" setup/extensions.py

  # Fix for calibre-0.8.58
  sed -i -e "s:#!usr:#!/usr:g" src/calibre/ebooks/markdown/extensions/meta.py

  sed -i -e 's:\(#!/usr/bin/env[ ]\+python$\|#![ ]/usr/bin/env[ ]\+python$\|#!/usr/bin/python$\):\12:g' \
    $(find . -regex ".*.py\|.*.recipe")

  LANG='en_US.UTF-8' python2 setup.py build
  # LANG='en_US.UTF-8' python2 setup.py resources
  LANG='en_US.UTF-8' python2 setup.py translations
}

package() {
  cd "${srcdir}/${pkgname}"

  patch -Np1 -i "${srcdir}/desktop_integration.patch"

  # More on desktop integration (e.g. enforce arch defaults)
  sed -i -e "/self.create_uninstaller()/,/os.rmdir(config_dir)/d" \
      -e "s|self.opts.staging_sharedir, 'man/man1'|self.opts.staging_root, 'usr/share/man/man1'|" \
      -e "s|manpath, prog+'.1'+__appname__+'.bz2'|manpath, prog+'.1'+'.bz2'|" \
      -e "s|old_udev = '/etc|old_udev = '${pkgdir}/etc|" \
      -e "s/^Name=calibre/Name=Calibre/g" src/calibre/linux.py

  # Fix the environment module location
  sed -i -e "s|(prefix=.*)|(prefix='$pkgdir/usr')|g" setup/install.py

  install -d "${pkgdir}/usr/lib/python2.7/site-packages"
  LANG='en_US.UTF-8' python2 setup.py install --root="${pkgdir}" --prefix=/usr \
    --staging-bindir="${pkgdir}/usr/bin" \
    --staging-libdir="${pkgdir}/usr/lib" \
    --staging-sharedir="${pkgdir}/usr/share"

  find "${pkgdir}" -type d -empty -delete

  # See http://lwn.net/SubscriberLink/465311/7c299471a5399167/
  rm -rf ${pkgdir}/usr/bin/calibre-mount-helper
  install -m 755 ${srcdir}/calibre-mount-helper "${pkgdir}/usr/bin"
}

Doing "yaourt calibre":

1 community/calibre 2.31.0-1
    Ebook management application
2 aur/calibre-git 2.1.0.44.g8659089-3 (1)
    Ebook management application, from git
3 aur/calibre-installer 0.3-1 (1)
    Automatically install and update the calibre standalone binaries

the last version is available, so why the version inside PGKBUILD is so old?


Should I edit also LANG sections inside PKGBUILD to set my native language?

[eschwartz]

Huh, I wonder where yaourt is pulling that from?

Either way, here is a (fixed) source tarball.

[plasmid]

Error during build:

$ makepkg -s
==> Making package: calibre 2.31.0-1 (Thu Jun 25 14:14:43 UTC 2015)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
  -> Found calibre-2.31.0.tar.xz
==> Validating source files with md5sums...
    calibre-2.31.0.tar.xz ... Passed
==> Extracting sources...
  -> Extracting calibre-2.31.0.tar.xz with bsdtar
==> Starting prepare()...
==> Removing existing $pkgdir/ directory...
==> Starting build()...

*
* Running build
*


####### Building extension dukpy #######

####### Building extension hunspell #######

####### Building extension _regex #######

####### Building extension speedup #######

####### Building extension certgen #######

####### Building extension html #######

####### Building extension tokenizer #######

####### Building extension _patiencediff_c #######

####### Building extension icu #######

####### Building extension sqlite_custom #######

####### Building extension chmlib #######

####### Building extension chm_extra #######

####### Building extension magick #######

####### Building extension lzx #######

####### Building extension freetype #######

####### Building extension woff #######

####### Building extension msdes #######

####### Building extension cPalmdoc #######

####### Building extension bzzdec #######

####### Building extension matcher #######

####### Building extension podofo #######

####### Building extension pictureflow #######
rm -f libpictureflow.so
g++ -Wl,-O1,--sort-common,--as-needed,-z,relro -Wl,--version-script=pictureflow.exp -Wl,-O1 -shared -o libpictureflow.so pictureflow.o sippictureflowcmodule.o sippictureflowPictureFlow.o sippictureflowFlowImages.o moc_pictureflow.o  -lQt5Widgets -lQt5Gui -lQt5Core -lGLESv2 -lpthread  
/usr/bin/ld: cannot find -lGLESv2
collect2: error: ld returned 1 exit status
Makefile:176: recipe for target 'libpictureflow.so' failed
make: *** [libpictureflow.so] Error 1
Error while executing: make -j4

Traceback (most recent call last):
  File "setup.py", line 99, in <module>
    sys.exit(main())
  File "setup.py", line 85, in main
    command.run_all(opts)
  File "/home/plasmid/calibre/src/calibre-2.31.0/setup/__init__.py", line 181, in run_all
    self.run_cmd(self, opts)
  File "/home/plasmid/calibre/src/calibre-2.31.0/setup/__init__.py", line 178, in run_cmd
    cmd.run(opts)
  File "/home/plasmid/calibre/src/calibre-2.31.0/setup/extensions.py", line 428, in run
    self.build(ext, dest)
  File "/home/plasmid/calibre/src/calibre-2.31.0/setup/extensions.py", line 450, in build
    return self.build_pyqt_extension(ext, dest)
  File "/home/plasmid/calibre/src/calibre-2.31.0/setup/extensions.py", line 640, in build_pyqt_extension
    self.check_call([make]+([] if iswindows else ['-j%d'%(cpu_count or 1)]))
  File "/home/plasmid/calibre/src/calibre-2.31.0/setup/extensions.py", line 502, in check_call
    subprocess.check_call(*args, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', '-j4']' returned non-zero exit status 2
==> ERROR: A failure occurred in build().
    Aborting...

Something related to PKGBUILD?

[eschwartz]

No, nothing is wrong with the PKGBUILD, but I think you don't have everything needed on your system. calibre's setup script is trying to build one of the extensions and is failing to find and link the "GLESv2" library.

It should be provided by the "mesa" package, which has the following dependency chain:
mesa --> mesa-libgl --> qt5-base --> pyqt5-common --> python2-pyqt5 --> calibre

i.e. a working qt5 should have this already.

[plasmid]

Missing mesa-libgl (due to a conflicting package).

Now everything works. Thank you so much for your preciuos help, I learned something more.

[eschwartz]

Sure, happy to help!

And if you ever have problems like that, I find pkgfile is an excellent tool for tracking down which package was supposed to supply the file you are missing.

The original error was:

g++ -Wl,-O1,--sort-common,--as-needed,-z,relro -Wl,--version-script=pictureflow.exp -Wl,-O1 -shared -o libpictureflow.so pictureflow.o sippictureflowcmodule.o sippictureflowPictureFlow.o sippictureflowFlowImages.o moc_pictureflow.o  -lQt5Widgets -lQt5Gui -lQt5Core -lGLESv2 -lpthread  
/usr/bin/ld: cannot find -lGLESv2
collect2: error: ld returned 1 exit status

So we are looking for a shared library to be linked ("-l") which is called "GLESv2", so that would be libGLESv2.so

[eschwartz@arch ~]$ pkgfile -v libGLESv2.so
extra/mesa 10.6.0-1                	/usr/lib/mesa/libGLESv2.so
extra/mesa-libgl 10.6.0-1          	/usr/lib/libGLESv2.so
extra/nvidia-304xx-libgl 304.125-1 	/usr/lib/libGLESv2.so
extra/nvidia-340xx-libgl 340.76-2  	/usr/lib/libGLESv2.so
extra/nvidia-340xx-utils 340.76-2  	/usr/lib/nvidia/libGLESv2.so
extra/nvidia-libgl 352.21-1        	/usr/lib/libGLESv2.so
extra/nvidia-utils 352.21-1        	/usr/lib/nvidia/libGLESv2.so
ARM-extra/mesa 10.6.0-1/files      	usr/lib/mesa/libGLESv2.so
ARM-extra/mesa-libgl 10.6.0-1/files	usr/lib/libGLESv2.so

(I added the ARM repos to pkgfile to search them, since I am on i686.)

[plasmid]

usefull information, thank you again

[shamanNS]

@eschwartz: Since you're so Linux (and Calibre) savvy I have one question for you:
-How difficult it would be to add/install Calibre to OpenELEC distro runing on "Android on a stick" ARM devices, and then use Calibre server?

[eschwartz]

A lot harder than one running on e.g. ArchLinuxArm.

The root of the problem is installing calibre's dependencies, and the lack of a package manager. AIUI, OpenELEC is stripped down to run very efficiently with just Kodi, and you aren't *expected* to install other things.

You'd have to manually compile all the dependencies by hand, using "./configure; make; sudo make install" and friends, struggle with the read-only system image, etc. -- OpenELEC really doesn't want you messing around with the system!



You could get a stripped down system that very efficiently runs only what you need it to, if you go the Gentoo route -- ArchLinux uses a (heavier) generic base system -- and also benefit from a package manager.
ArchLinuxArm or Gentoo, either way will require more effort to set up.


Or use the debian-based XBian or OSMC, and pull their (old?) calibre packages.

[eschwartz]

Note to all readers -- rather than patching the ArchLinux PKGBUILD as I recommended to the OP, you can now also use my calibre-git package from the AUR. It does the Right Thing, and hey, you're building from source anyway.