Quote:
Originally Posted by davidfor
Looking at the code, I agree that the loop over the futures should be inside the with. But, I'm not sure what that would help.
|
The threading code is not easy to understand!
That said, when the ThreadPoolExecutor instance is garbage collected, which can happen any time after the "with" exits, some of the internal structure for the pool goes away. This is detected by the threads using weakrefs. It seems that in this case a thread can terminate instead of doing the next task on the queue. If this is true, and if there are more tasks than threads (more files than threads), then sometimes a task will be skipped. Putting the future.result() calls inside the "with" avoids this possibility.
I won't stake my reputation on this analysis.
