CBZ conversion crashes with large images

[ratinox]

I would provide a conversion log but there isn't one. ebook-convert -d crashes during the rendering phase without writing anything but the README.txt. This is large (500MB-1GB) CBZ files with high resolution PNG images (3550x4950 up to 10115x14450). Humble Bundle can deliver quite high resolution scans. The Python traceback isn't much help to me:

5% Rendered /tmp/calibre_3.39.1_tmp_ULi06y/q46NYH_comic_extract/v01/004.png
5% Rendered /tmp/calibre_3.39.1_tmp_ULi06y/q46NYH_comic_extract/v01/063.png
Traceback (most recent call last):
  File "/usr/bin/ebook-convert", line 20, in <module>
    sys.exit(main())
  File "/usr/lib/calibre/calibre/ebooks/conversion/cli.py", line 391, in main
    plumber.run()
  File "/usr/lib/calibre/calibre/ebooks/conversion/plumber.py", line 1106, in run
    accelerators, tdir)
  File "/usr/lib/calibre/calibre/customize/conversion.py", line 244, in __call__
    log, accelerators)
  File "/usr/lib/calibre/calibre/ebooks/conversion/plugins/comic_input.py", line 182, in convert
    pages = self.get_pages(fname, cdir)
  File "/usr/lib/calibre/calibre/ebooks/conversion/plugins/comic_input.py", line 146, in get_pages
    self.report_progress, tdir2)
  File "/usr/lib/calibre/calibre/ebooks/comic/input.py", line 271, in process_pages
    job.log_file.read())
Exception: Failed to process comic:

I can downscale with ImageMagick and then feed into ebook-convert but IM requires increased memory, map, area etc. limits in policy.xml from Debian's defaults so I'm wondering if there's a memory allocation limit I'm hitting in Calibre. 64-bit version if it matters.

[kovidgoyal]

First of use the official binaries, not the debian package. Then if you are still getting a crash, get a backtrace. I'm not familiar with how one does it on debian, but on a systemd based distro, something like

echo bt | coredumpctl gdb

And calibre has no allocation limits, it just uses system malloc

[ratinox]

Originally Posted by kovidgoyal

First of use the official binaries, not the debian package. Then if you are still getting a crash, get a backtrace. I'm not familiar with how one does it on debian, but on a systemd based distro, something like

echo bt | coredumpctl gdb

And calibre has no allocation limits, it just uses system malloc

Will try that and let you know how it goes. Thank you.

[ratinox]

Tried current and it doesn't crash! Yay!
It's a memory exhaustion problem. Boo.

Rendered /tmp/calibre_3.48.0_tmp_gttcnB/UUtd8B_comic_extract/v01/048.png
Failed /tmp/calibre_3.48.0_tmp_gttcnB/UUtd8B_comic_extract/v01/049.png
Failed /tmp/calibre_3.48.0_tmp_gttcnB/UUtd8B_comic_extract/v01/050.png
Failed /tmp/calibre_3.48.0_tmp_gttcnB/UUtd8B_comic_extract/v01/051.png
Failed /tmp/calibre_3.48.0_tmp_gttcnB/UUtd8B_comic_extract/v01/052.png
Failed /tmp/calibre_3.48.0_tmp_gttcnB/UUtd8B_comic_extract/v01/053.png
Failed /tmp/calibre_3.48.0_tmp_gttcnB/UUtd8B_comic_extract/v01/054.png
Failed /tmp/calibre_3.48.0_tmp_gttcnB/UUtd8B_comic_extract/v01/055.png
Failed /tmp/calibre_3.48.0_tmp_gttcnB/UUtd8B_comic_extract/v01/056.png
Failed /tmp/calibre_3.48.0_tmp_gttcnB/UUtd8B_comic_extract/v01/057.png
Failed /tmp/calibre_3.48.0_tmp_gttcnB/UUtd8B_comic_extract/v01/058.png
Rendered /tmp/calibre_3.48.0_tmp_gttcnB/UUtd8B_comic_extract/v01/059.jpg
Rendered /tmp/calibre_3.48.0_tmp_gttcnB/UUtd8B_comic_extract/v01/060.jpg
Rendered /tmp/calibre_3.48.0_tmp_gttcnB/UUtd8B_comic_extract/v01/061.jpg
Rendered /tmp/calibre_3.48.0_tmp_gttcnB/UUtd8B_comic_extract/v01/062.jpg
QImage: out of memory, returning null image
Failed /tmp/calibre_3.48.0_tmp_gttcnB/UUtd8B_comic_extract/v01/063.png

So looks like my options are to rewrite the rendering process to use RAM more efficiently and don't do that. I'll stick with "don't do that" for now.

Thanks.