You cannot use multiprocessing safely in native code programs (such as calibre) that embed a python interpreter. In fact, you cant use it safely at all, it is one of the worst designed modules in the standard library. Read its docs, in particular
The 'spawn' and 'forkserver' start methods cannot currently be used with “frozen” executables (i.e., binaries produced by packages like PyInstaller and cx_Freeze) on Unix. The 'fork' start method does work.
And fork is broken in any program that uses threading anyway. So multiprocessing is a complete no go.
Again, let me ask, what it is you want to actually do. calibre already has the ability to run worker processes in several ways. It does not however touch multiprocessing with ten foot barge pole.
|