[chaley]

Originally Posted by kovidgoyal

I/O operations do use multiple cores, since python releases the GIL during a read/write. My concern is the you will need to have a loop that looks like

had_operation = False
for socket in connections:
   if socket.poll():
      #handle the read in a relatively non blocking manner
      had_operation = True
if not had_operation:
   #ensure release of the GIL
    time.sleep(0.01)

That just seems beyond ugly and I really dont see it being performant

If you are using a thread, then the poll goes away. You would do something like

try:
 with open(x) as f:
   d = f.read(someAmount)
   outpipe.write(d)
finally:
 outpipe.close()

The GIL will be released and threads will switch if the write to outpipe blocks because the queue is full. If the reader goes away, the write to outpipe will throw an exception. This should perform very well.