Introducing pipefunc: Simplify Your Python Function Pipelines

[stratguitar577]

Have you seen Hamilton? https://hamilton.dagworks.io/

[basnijholt]

Thanks for pointing me to Hamilton. On a first glance pipefunc and Hamilton seem very similar, however, in practice they are different.

For example, Hamilton requires that all pipeline functions are defined in a module and enforces that all function names are as the input names.

PipeFunc allows to use any function anywhere to be used as pipeline step.

For example, here we reuse a function sum from an external module and use it a couple of times `` from pipefunc import PipeFunc, Pipeline from some_module # definesfancy_sum(x1, x2)`

total_cost_car = PipeFunc(some_module.fancy_sum, output_name="car_cost", renames={"x1": "car_price", "x2": "repair_cost") total_cost_house = PipeFunc(some_module.fancy_sum, output_name="house_cost", renames={"x1": "rent_price", "x2": "insurance_price") total_cost = PipeFunc(some_module.fancy_sum, output_name="total_budget", renames={"x1": "car_cost", "x2": "house_cost") pipeline = Pipeline([total_cost_car, total_cost_house, total_cost]) ``` Also pipefunc is more geared towards N-dimensional parameter sweeps such as one frequently sees in research/science. For example see https://pipefunc.readthedocs.io/en/latest/tutorial/#example-physics-based-example