Originally Posted by hawhill
While the garbage collection could be tuned, I don't think it would help. The actual rendering of the PDF seems to be the culprit, and that is part of the (C language) dependent library MuPDF. Some corner cases will probably always trigger a high memory usage, seems to be one of them.
I know your using LuaJIT, but it should have maintained compatibility with the standard lauconf.h (one can hope anyway).
LUAI_GCPAUSE is set at 200 out-of-the box - -
redefine as: 110, re-build.
From: Lua Programming Gems, chapter 26
Give that a try - it may leave enough memory room for the C backends to work.