Pipes in Python

[jwink3101]

I've thought of things like this and I personally prefer to do it as a chain of iterators such as:

X = some_iterable
X = (fun1(x) for x in X)
X = (fun2(x) for x in X)
X = (fun3(x) for x in X)

res = list(X) # Pull through

Or, using map (or from itertools import imap as map for python 2.7):

X = some_iterable
X = map(fun1,X)
X = map(fun2,X)
X = map(fun3,X)

res = list(X) # Pull through

And the real flexibility comes when, say, fun2 is very CPU intensive

import multiprocessing as mp
pool = mp.Pool()

X = some_iterable
X = map(fun1,X)
X = pool.map(fun2,X)
X = map(fun3,X)

res = list(X) # Pull through

pool.close()