[kovidgoyal]

Originally Posted by chaley

I don't think the GIL is really an issue. Python will switch between the threads as it wishes, distributing the time to do the copy over several context switches. Yes, multiple cores are not used, but this is true even if the file is copied. The work gets done in the same amount of time, but threaded solutions better spread that work over time.If the problem is not producer-consumer, then you are absolutely correct. If it is P/C, then I am not convinced.

I/O operations do use multiple cores, since python releases the GIL during a read/write. My concern is the you will need to have a loop that looks like

had_operation = False
for socket in connections:
   if socket.poll():
      #handle the read in a relatively non blocking manner
      had_operation = True
if not had_operation:
   #ensure release of the GIL
    time.sleep(0.01)

That just seems beyond ugly and I really dont see it being performant

One thing I don't know is what percentage of operations are what. Readers are P/C, but modifiers are not.

We can effectively make them all readers by requiring the writes to be run in process.