Quote:
Originally Posted by kovidgoyal
@chaley: A pipe sounds interesting, I've never implemented one, I guess it is basically a FIFO Queue that only reads a few KB at a time into the queue. The question is what is the best architecture for filling the buffer? A separate thread or just fill when emptied in the emptying thread? The tradeoff, I guess, is between amount of time file handle has to be help open vs. efficiency?
|
Pipes are indeed fifo queues, and are very easy to use. os.pipe returns the two file descriptors that act just as you would expect.
The best way to write to pipes is to use a thread. Normally, pipe writing will block until the reader takes data, and this can be a pain to work around, usually requiring the use of "select". I don't know if python has a good/usable implementation of select.
If you use a thread, then both open file handling and feeding the pipe are natural, using a "with ...:" construction. The same thing can support locks by using a "with" inside a "with".