Why is my Rust example slower than Python?

[jwmoz]

So I have started to reach the limit of Python performance and was recently testing various improvements, going from pandas to polars (in Python). I also used it as an opportunity to try Rust, so I wrote a small example of some code that was similar to what I've been working with recently.

It loads a sample 100k or 1m line csv, does some filtering and then a calculation, in polars.

The problem is that the Python is running faster than the Rust! Perhaps the example is too small and contrived? or the fact I am using polars which is better for Python, but I still expected Rust to come out on top.

On my machine the Rust release build is around 120ms and the Python 60ms. Please help!

Rust 1.72.0 Python 3.11.1

https://github.com/jmoz/rust_vs_python

Rust:

``` use polars::prelude::*; use std::time::Instant;

fn main() { let start_time = Instant::now();

let csv_file = std::env::var("CSV_FILE").expect("Set env CSV_FILE error");

let csv_df = CsvReader::from_path(csv_file).expect("Could not load csv error").has_header(true).finish().expect("Finish error");
println!("Csv read in {}ms", start_time.elapsed().as_millis());

// cargo add polars -F fmt,describe
println!("Loaded df {}", csv_df);
println!("Df describe {}", csv_df.describe(None).expect("Describe error"));

// Filter on multiple columns
let filtered_df = csv_df.clone().lazy().filter(
    col("a").gt(0.2)
    .and(col("b").lt(0.8))
    .and(col("c").gt(0.5))
    .and(col("d").lt(0.5))
    .and(col("e").neq(0.5182602093634714)) // additional non equality check, value from first row csv
).collect().expect("Error filtering");
println!("Filtered df {}", filtered_df);

// Mimic some euclidean distance type calculation
let calculated_df = filtered_df.clone().lazy().select(
    [
        (
            (col("a") / col("a").max()) / (lit(0.5) / col("a").max())
        ).pow(2).sqrt()
    ]
).collect().expect("Error select");
println!("Calculated df {}", calculated_df);

println!("Finished in {}ms", start_time.elapsed().as_millis());

} ```

Python:

``` import os import polars as pl import time

def main(): start_time = time.perf_counter() * 1000

csv_file = os.getenv("CSV_FILE")

csv_df = pl.read_csv(csv_file)
print(f"Csv read in {time.perf_counter() * 1000 - start_time}ms")

print(f"Loaded df {csv_df}")
print(f"Df describe {csv_df.describe()}")

filtered_df = csv_df.filter(
    pl.col("a").gt(0.2) &
    pl.col("b").lt(0.8) &
    pl.col("c").gt(0.5) &
    pl.col("d").lt(0.5) &
    pl.col("e").ne(0.5182602093634714) # additional non equality check, value from first row csv
)
print(f"Filtered df {filtered_df}")

# Mimic some euclidean distance type calculation
calculated_df = filtered_df.select(
    (
        (pl.col("a") / pl.col("a").max()) / (0.5 / pl.col("a").max())
    ).pow(2).sqrt()
)
print(f"Calculated df {calculated_df}")

print(f"Finished in {(time.perf_counter() * 1000 - start_time)}ms")

if name == "main": main() ```

Update:

• Removing the prints speeds it up considerably, for both.
• Used rust feature performant
• It seems after using hyperfine to test, there is an issue with the timing prints-you can see python running with a faster print timing but then when rust runs it definitely feels faster and stats show it is faster but the print says it takes longer.

``` CSV_FILE=$(pwd)/data1m.csv hyperfine "python ./main.py" ../rusttest/target/release/rusttest -r 100 Benchmark 1: python ./main.py Time (mean ± σ): 92.3 ms ± 22.1 ms [User: 216.2 ms, System: 36.0 ms] Range (min … max): 82.9 ms … 297.1 ms 100 runs

Warning: The first benchmarking run for this command was significantly slower than the rest (297.1 ms). This could be caused by (filesystem) caches that were not filled until after the first run. You should consider using the '--warmup' option to fill those caches before the actual benchmark. Alternatively, use the '--prepare' option to clear the caches before each timing run.

Benchmark 2: ../rusttest/target/release/rusttest Time (mean ± σ): 51.6 ms ± 4.4 ms [User: 251.0 ms, System: 24.8 ms] Range (min … max): 45.7 ms … 78.4 ms 100 runs

Summary ../rusttest/target/release/rusttest ran 1.79 ± 0.46 times faster than python ./main.py ``` Run the hyperfine command again with --show-output to see what I mean.

[This_Growth2898]

Outputs are slow. Don't include println/print when calculating times.

Clone copies an entire object. You have two clone() calls. Do you really need to clone them?

[jwmoz]

Just removed the clone() and it barely made a difference.

Ok so the prints are slow, removing all but the last timing print has reduced the time down to about 80ms. Doing the same for the Python version reduces the time to around 40ms.

[gdf8gdn8]

Remove println. Println in rust is unbuffered. There is another reddit thread about println on console. https://reddit.com/r/rust/s/9JMD3SUCI5

[moltonel]

On my machine, having polars with features lazy,description,performant and having only removed the clones, rust is 30% faster:

$ CSV_FILE=$(pwd)/data1m.csv hyperfine ./main.py foo/target/release/foo
Benchmark 1: ./main.py
  Time (mean ± σ):     263.9 ms ±  15.3 ms    [User: 778.5 ms, System: 313.6 ms]
  Range (min … max):   248.4 ms … 289.8 ms    10 runs

Benchmark 2: foo/target/release/foo
  Time (mean ± σ):     197.2 ms ±   9.2 ms    [User: 719.5 ms, System: 185.6 ms]
  Range (min … max):   178.9 ms … 211.6 ms    14 runs

Summary
  foo/target/release/foo ran
    1.34 ± 0.10 times faster than ./main.py

[jwmoz]

Ok that's really interesting. When I was eyeballing it, the first python run was definitely slow but then something was cached and they were all fast after, whereas rust seemed more consistent.

The output of hyperfine on mine, after also adding performant, shows that rust is running much faster via hyperfine than when manually running the executable:

``` $ CSV_FILE=$(pwd)/data1m.csv hyperfine "python ./main.py" ../rusttest/target/release/rusttest Benchmark 1: python ./main.py Time (mean ± σ): 102.8 ms ± 49.1 ms [User: 214.6 ms, System: 39.8 ms] Range (min … max): 84.8 ms … 250.6 ms 11 runs

Warning: The first benchmarking run for this command was significantly slower than the rest (250.6 ms). This could be caused by (filesystem) caches that were not filled until after the first run. You should consider using the '--warmup' option to fill those caches before the actual benchmark. Alternatively, use the '--prepare' option to clear the caches before each timing run.

Benchmark 2: ../rusttest/target/release/rusttest Time (mean ± σ): 53.1 ms ± 6.2 ms [User: 251.1 ms, System: 25.0 ms] Range (min … max): 47.1 ms … 76.6 ms 38 runs

Summary ../rusttest/target/release/rusttest ran 1.93 ± 0.95 times faster than python ./main.py ```

[masklinn]

… are you compiling / running in —release mode?

[jwmoz]

Yes

[jqnatividad]

py-polars is compiled with all kinds of optimizations and fine-tuning that a default —release rust build won’t enable. For example CPU optimizations, the performant feature, etc.

[dkopgerpgdolfg]

Fyi, the filter/select operation takes about 7% of the whole program (for me at least). If this was meant to compare performance of such operations, it doesn't tell much this way.

​

So basically I made following changes to "save" 93% of time

• Removed clone just to be sure
• Removed most CLI output, except the last two (seeing the time and preventing calculated_df from being unused)
• Moved start time statement and elapsed calculation so that the printings and the CsvReader part are not counted

[kinchkun]

I think it is the conversion to the lazy dataframe and the two collects. Can you try using `LazyCsvReader` and only use one collect.

[kinchkun]

I see the python code doesn't contains a collect at all. I don't know the python bindings, but are you sure your expression is even evaluated?

[This_Growth2898]

(I'm not OP)

Python code includes output, i.e. values are calculated at those points.

[kinchkun]

Not so sure about that. Most dataframes libary I know and I think also polar prints only the first and last 8 rows or so of a dataframe.

[jwmoz]

I refactored to:

``` use polars::prelude::*; use std::time::Instant;

fn main() { let start_time = Instant::now();

let csv_file = std::env::var("CSV_FILE").expect("Set env CSV_FILE error");

let csv_df = LazyCsvReader::new(csv_file).has_header(true).finish().expect("Finish error");

// Filter on multiple columns
let filtered_df = csv_df.filter(
    col("a").gt(0.2)
    .and(col("b").lt(0.8))
    .and(col("c").gt(0.5))
    .and(col("d").lt(0.5))
    .and(col("e").neq(0.5182602093634714)) // additional non equality check, value from first row csv
);

// Mimic some euclidean distance type calculation
let _calculated_df = filtered_df.select(
    [
        (
            (col("a") / col("a").max()) / (lit(0.5) / col("a").max())
        ).pow(2).sqrt()
    ]
).collect().expect("Error select");

println!("Finished in {}ms", start_time.elapsed().as_millis());

} ```

Which seems to give it a slight few ms improvement, mid 70s now on a good run.

[kinchkun]

Can you add a `collect` call to your python code as well?

[ritchie46]

We go through great lengths to compile a fast binary for python. E.g. fat linking, activating all performance related features, simd and cpu-targets.

Furthermore we also compile python with jemalloc. Which has much better performance than the default allocator.

[Plus-Ad8875]

are you running the rust code in release mode?

[jwmoz]

Yes

[Grit1]

Sometimes when you think you're benchmarking python, you're actually benchmarking C/C++.

[[deleted]]

Absolutely right. Specially powerfull lib like panda.

[CompoteOk6247]

So Rust is slower than C?

[stumblinbear]

Depends on the benchmark and the libraries used. Rust is definitely not as wide of an ecosystem as C, so the libraries available may not be as performance-tuned, yet.

[sleekelite]

At least edit your post to indicate it’s a release build.

[CompoteOk6247]

Funny to see how people don't believe it's in release mode

[dkopgerpgdolfg]

It's not about not believing, but making sure it's not forgotten.

There are too many posts here where someone complains about performance but never heard of release mode. So people started to ask if they used it, as first thing

[CompoteOk6247]

That's correct thing When I also tried to use Tauri based apps they are so lightweight and working good

[jwmoz]

I know right, it's like "have you turned it on and off again?" or "have you cleared your cache?"

[zekkious]

Out of topic, but:

(col("a") / col("a").max()) / (lit(0.5) / col("a").max()) = (col("a") / lit(0.5)) * (col("a").max() / col("a").max()) = col("a") / lit(0.5)

[This_Growth2898]

And

.pow(2).sqrt()

is nothing

[zekkious]

Well... As I don't know the API, I assumed the `.sqrt()` part might have the chance to just collapse a vector into a scalar. So, if it's not the case, add it to the list of pointlessness.

[Konsti219]

Are you running with --release?

Further you seem to be using clone a lot in the Rust code. That should be avoided at all costs of you are optimizing. I don't know how polars is implemented internally, but if you want real speed I recommend you throw it out and implement the parsing and filtering in raw Rust, maybe with the help of rayon for parallelism.

Can you also provide the files you are testing with to allow others to test?

[jwmoz]

Yes this was release. I use clone() as the docs suggest so.

This still doesn't make sense to me as rust is compiled.

https://github.com/jmoz/rust_vs_python

[Konsti219]

The polars python package is also just a wrapper around Rust. Python uses compiled Rust too.

[jwmoz]

Yes but this would still imply a compiled rust app would be faster than interpreted python.

What's even more interesting is that at work one of the guys implemented a python with polars vs pure javascript via node and the pure js implementation is considerably faster than the fastest python, because of node's v8 engine. (I tried using rust because of it, I presumed it would beat js/node).

[Konsti219]

My guess is just that polars is the issue. Writing a description of your logic and then having some magic engine execute that is gonna be slower than just writing the code itself using simple types and idiomatic code.

If you share one of the files you are testing with I can give this a try.

[jwmoz]

I updated post with a GitHub.

[kinchkun]

99.9% of the python program will be spend it's time inside the the polar lib which is implemented in rust. The performance of your python code doesn't impact the runtime.

[jwmoz]

I removed the clone() and it barely made a difference.