I tried using the multiprocessing Queue class, but it doesn't seem possible to pass it through the IPC. From what I understand what the IPC module does:
Job -> Worker -> ConnectedWorker -> Sends the kwargs to the worker via pipe
But the only way to send a Queue is when you spawn the child process, otherwise you get an error: Queue objects should only be shared between processes through inheritance.
I will try to use a Dispatcher with a Job result to see if that works.
|