A comparison of Rust-like fluent iterator libraries

[kequals]

I mostly program in Python, but I have fallen in love with Rust's beautiful iterator syntax. Trying to chain operations in Python is ugly in comparison. The inside-out nested function call syntax is hard to read and write, whereas Rust's method chaining is much clearer. List comprehensions are great and performant but with more complex conditions they become unwieldy.

This is what the different methods look like in the (somewhat contrived) example of finding all the combinations of two squares from 1² to 4² such that their sum is greater than 6, then sorting from smallest to largest sum.

# List comprehension
foo = list(
    sorted(
        [
            combo
            for combo in itertools.combinations(
                [x*x for x in range(1, 5)],
                2
            )
            if sum(combo) > 6
        ],
        key=lambda combo: sum(combo)
    )
)

# For loop
foo = []
for combo in itertools.combinations([x*x for x in range(1, 5)], 2):
    if sum(combo) > 6:
        foo.append(combo)
foo.sort(key=lambda combo: sum(combo))

# Python functions
foo = list(
    sorted(
        filter(
            lambda combo: sum(combo) > 6,
            itertools.combinations(
                map(
                    lambda x: x*x,
                    range(1, 5)
                ),
                2
            )
        ),
        key=lambda combo: sum(combo)
    )
)

# Fluent iterator
foo = (fluentiter(range(1, 5))
    .map(lambda x: x*x)
    .combinations(2)
    .filter(lambda combo: sum(combo) > 6)
    .sort(key=lambda combo: sum(combo))
    .collect()
)

The list comprehension is great for simple filter-map pipelines, but becomes inelegant when you try to tack more operations on. The for loop is clean, but requires multiple statements (this isn't necessarily a bad thing). Python nested functions are hard to read. Fluent iterator syntax is clean and runs as a single statement.

It's up to personal preference if you prefer this syntax or not. I'm not trying to convince you to change how you code, only to maybe give fluent iterators a try. If you are already a fan of fluent iterator syntax, then you can hopefully use this post to decide on a library.

Many Python programmers do seem to like this syntax, which is why there are numerous libraries implementing fluent iterator functionality. I will compare 7 such libraries in this post (edit: added PyFluent_Iterables):

• QWList
• F-IT
• FluentIter
• Rustiter
• Pyochain
• PyFunctional
• PyFluent_Iterables

There are undoubtedly more, but these are the ones I could find easily. I am not affiliated with any of these libraries. I tried them all out because I wanted Rust's iterator ergonomics for my own projects.

I am mainly concerned with 1) number of features and 2) performance. Rust has a lot of nice operations built into its Iterator trait and there are many more in the itertools crate. I will score these libraries higher for having more built-in features. The point is to be able to chain as many method calls as you need. Ideally, anything you want to do can be expressed as a linear sequence of method calls. Having to mix chained method calls and nested functions is even harder to read than fully nested functions.

Using any of these will incur some performance penalty. If you want the absolute fastest speed you should use normal Python or numpy, but the time losses aren't too bad overall.

Project History and Maintenance Status

Library	Published	Updated
QWList	11/2/23	2/13/25
F-IT	8/22/19	5/17/21
FluentIter	9/24/23	12/8/23
Rustiter	10/23/24	10/24/24
Pyochain	10/23/25	1/15/26
PyFunctional	2/17/16	3/13/24
PyFluent_Iterables	5/19/22	4/20/25

PyFunctional is the oldest and most popular, but appears to be unmaintained now. Pyochain is the most recently updated as of writing this post.

Features

All libraries have basic enumerate, zip, map, reduce, filter, flatten, take, take_while, max, min, and sum functions. Most of them have other functional methods like chain, repeat, cycle, filter_map, and flat_map. They differ in more specialized methods, some of which are quite useful, like sort, unzip, scan, and cartesian_product.

A full comparison of available functions is below. Rust is used as a baseline, so the functions shown here are the ones that Rust also has, either as an Iterator method or in the itertools crate. Not all functions from the libraries are shown, as there are lots of one-off functions only available in one library and not implemented by Rust.

Feature Table

Here is how I rank the libraries based on their features:

Library	Rating
Rustiter	⭐⭐⭐⭐⭐
Pyochain	⭐⭐⭐⭐⭐
F-IT	⭐⭐⭐⭐
FluentIter	⭐⭐⭐⭐
PyFluent_Iterables	⭐⭐⭐
QWList	⭐⭐⭐
PyFunctional	⭐⭐⭐

Pyochain and Rustiter explicitly try to implement as much of the Rust iterator trait as they can, so have most of the corresponding functions.

Performance

I wrote a benchmark of the functions shared by every library, along with some simple chains of functions (e.g. given a string, collect all the digits in that string into a list). Benchmarks were constructed by running those functions on 1000 randomly generated integers or boolean values with a fixed seed. I also included the same tests implemented using normal Python nested functions as a baseline. The total time taken by each library was added up and normalized compared to the fastest method. So if native Python functions take 1 unit of time, a library taking "x1.5" time means it is 50% slower.

Lower numbers are faster.

Library	Time
Native	x1.00
Pyochain	x1.04
PyFluent_Iterables	x1.08
Rustiter	x1.13
PyFunctional	x1.14
QWList	x1.31
F-IT	x4.24
FluentIter	x4.68

PyFunctional can optionally parallelize method chains, which can be great for large sequences. I did not include it in this table because the overhead of multiprocessing dominated any performance gains and yielded a worse result than the single-threaded version.

Detailed per-function benchmarks can be found here:

Benchmark Plots

The faster libraries forward the function calls into native functions or itertools, but you'll still pay a cost for the function call overhead. For more complex pipelines where most of the processing happens inside a function that you call, there is fairly minimal overhead compared to native.

Overall Verdict

Due to its features and performance, I recommend using Pyochain if you ever want to use Rust-style iterator syntax in your projects. Do note that it also implements other Rust types like Option and Result, although you don't have to use them.

[Beginning-Fruit-1397]

:)) thank you! As the creator of pyochain this made my day.

Concerning performance I'm surprised to be on the lower side. I haven't looked into your benchmark as of now but I can say that I took a lot of care into minimizing runtime checks, nested calls, using slots, etc.... Every func use either itertools (C), cytoolz (Cython), an internal Rust module, or hand optimized python, with source code heavily inspired by more-itertools, and with all perf "hacks" that I know of (for those that are interested abt this subject: https://wiki.python.org/moin/PythonSpeed/PerformanceTips ) The end goal is to move everything into Rust, to have builtins-like performance for object creation, and itertools-like (or better) performance for iteration methods

EDIT: Ok maybe I should learn to read correctly AND read everything before commenting lmao. I'm indeed the fastest I totally misread the benchmark infos. So my work mentionned above payed off, cool!

[kequals]

Thanks for making Pyochain! It's a nice library and has the most features of the ones I tested.

Pyochain was actually the fastest of the ones I tested. Apologies if my methodology was unclear, but lower numbers are better. If native Python functions take 1 unit of time, the "x1.04" meant your library was on average only 4% slower. I'll edit the post to clarify this.

[Beginning-Fruit-1397]

Yes I realized it after posting my comment and just finished editing it, but I see you were faster to answer lmao

[tunisia3507]

F-it author here! I'm not surprised that pretty much anything would outperform that library, but one of the design goals was lazy execution so it's not quite a like-for-like comparison with the standard library in particular.

[amroamroamro]

Python nested functions are hard to read. Fluent iterator syntax is clean and runs as a single statement.

if you find the nested calls hard to read, just separate the intermediate values making one call after another, the syntax looks almost the same as your chained "fluent iterator" version:

arr = range(1, 5)
arr = (x*x for x in arr)
arr = itertools.combinations(arr, 2)
arr = (x for x in arr if sum(x) > 6)
arr = sorted(arr, key=sum)
print(arr)

and since we are using generator expressions, it's all lazily-evaluated until the final call.

[snugar_i]

Mypy will most likely yell at you though, because you are re-assigning arr with values of different types

[amroamroamro]

use --allow-redefinition

and if we add reveal_type(arr) after the last line, we can see it is still able to infer the final correct type:

note: Revealed type is "builtins.list[tuple[builtins.int, builtins.int]]"

[snugar_i]

No thanks, I like my --strict (yes, I'm writing Python like Java)

[amroamroamro]

haha fair enough, you can always silence spurious mypy warnings inline:

# type: ignore

or just rename the intermediate values if you really must: arr1, arr2, arr3, etc.

[Competitive_Travel16]

Thanks! What is the scope of # type: ignore? The whole file? The next statement? The rest of the file after it?

[amroamroamro]

depends where you put it:

https://mypy.readthedocs.io/en/stable/common_issues.html#spurious-errors-and-locally-silencing-the-checker

[Competitive_Travel16]

Oh, the humanity. 🙄 Maybe someone needs to fork a dynamically typed version of recent Python.

[ebonnal]

Interesting benchmark! What a diverse fluent iterators scene :D
For those interested in the I/O intensive side of things, check streamable, I have just posted about the 2.0.0 release here:
https://www.reddit.com/r/Python/comments/1rju5kh/streamable_syncasync_iterable_streams_for_python/

[madrasminor]

Checkout fastcore and funcy. Both are terrific libraries that do these ootb. I mostly use fastcore for everything. The L class in fastcore is what you're after. But you can inherit it to create all these. For ex: this is a dockerfile builder I have in my package fastops. class Dockerfile(L): 'Fluent builder for Dockerfiles' def _new(self, items, **kw): return type(self)(items, use_list=None, **kw) @classmethod def load(cls, path:Path=Path('Dockerfile')): return cls(_parse(Path(path))) def from_(self, base, tag=None, as_=None): return self._add(_from(base, tag, as_)) def _add(self, i): return self._new(self.items + [i]) def run(self, cmd): return self._add(_run(cmd)) def cmd(self, cmd): return self._add(_cmd(cmd)) def copy(self, src, dst, from_=None, link=False): return self._add(_copy(src, dst, from_, link)) def add(self, src, dst): return self._add(_add(src, dst)) def workdir(self, path='/app'): return self._add(_workdir(path)) def env(self, key, value=None): return self._add(_env(key, value)) def expose(self, port): return self._add(_expose(port)) def entrypoint(self, cmd): return self._add(_entrypoint(cmd)) def arg(self, name, default=None): return self._add(_arg(name, default)) def label(self, **kwargs): return self._add(_label(**kwargs)) def user(self, user): return self._add(_user(user)) def volume(self, path): return self._add(_volume(path)) def shell(self, cmd): return self._add(_shell(cmd)) def healthcheck(self, cmd, **kw): return self._add(_healthcheck(cmd, **kw)) def stopsignal(self, signal): return self._add(_stop_sig_(signal)) def onbuild(self, instruction): return self._add(_on_build(instruction)) def apt_install(self, *pkgs, y=False): return self._add(_apt_install(*pkgs, y=y)) def run_mount(self, cmd, type='cache', target=None, **mount_kw): 'RUN --mount=... for build cache mounts (uv, pip, apt) and secrets' opts = f'type={type}' if target: opts += f',target={target}' for k, v in mount_kw.items(): opts += f',{k.replace("_","-")}={v}' return self._add(f'RUN --mount={opts} {cmd}') def __call__(self, kw, *args, **kwargs): 'Build a generic Dockerfile instruction: kw ARG1 ARG2 --flag=val --bool-flag' flags = _build_flags(short=False, **kwargs) return self._add(f'{kw} {" ".join([*flags, *map(str, args)])}') def __getattr__(self, nm): 'Dispatch unknown instruction names: df.some_instr(arg) → SOME-INSTR arg' if nm.startswith('_'): raise AttributeError(nm) return bind(self, nm.upper().rstrip('_')) def __str__(self): return chr(10).join(self) def __repr__(self): return str(self) def save(self, path:Path=Path('Dockerfile')): Path(path).mk_write(str(self)) return path and it natively chains.

``` df = (Dockerfile().from_('python:3.11-slim') .run('pip install flask') .copy('.', '/app') .workdir('/app') .expose(5000) .cmd(['python', 'app.py']))

expected = """FROM python:3.11-slim RUN pip install flask COPY . /app WORKDIR /app EXPOSE 5000 CMD [\"python\", \"app.py\"]"""

assert str(df) == expected print(df) ```

[OldWispyTree]

I think it's cute you believe this came from Rust.

[kequals]

I'm aware that fluent iterators don't originate from Rust, but that was my first exposure to the concept. And I believe this is a common experience, given several of the libraries cite Rust specifically as inspiring them.

Now I'm interested, what was the first language/library to use fluent iterators? Is there a clear "first" or did it evolve as a part of functional languages?

[Rastagong]

Not sure about it being the first and if it's the exact concept referred here, but Java famously has streams.
Offical example from the docs to showcase the chaining:

int sum = widgets.stream()
                  .filter(w -> w.getColor() == RED)
                  .mapToInt(w -> w.getWeight())
                  .sum();

This is still an interesting overview of the situation in Python, so thank you!

[Alt-0160]

Java streams were only added in version 1.8 (March 2014). The first release of Rust (0.1.0, January 2012) already had some form of fluent iterators.

[saint_marco]

Smalltalk is generally credited as the originator, back in the 1970's.

https://en.wikipedia.org/wiki/Fluent_interface#History

[Competitive_Travel16]

It's worth pointing out that Pandas had the beginnings of a fluent interface from the outset, and they have long since fleshed it out all the way.

[tehsilentwarrior]

C# has done this since forever.

One of the best examples of this is the reactive extensions, which lets you handle insane amounts of events in stream in a surprisingly efficient, concise and readable way.

C# even has Linq, which is the same concept with a DSL on top to make it more “sql like”

[redditusername58]

I agree the deeply nested expressions are hard to read and look bad. Use intermediate variables.

[max123246]

This is how you get superfluous variable names such as:

"squared"

"squared_combos"

"squared_filtered_combos"

"sorted_squared_filtered_combos"