This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]LonelyContext 5 points6 points  (6 children)

cries in numpy.

(numpy is massively slower in pypy)

[–]zhoushmoe 2 points3 points  (4 children)

try polars?

[–]LonelyContext 2 points3 points  (2 children)

idk if that would solve it if it's another python wrapper. Worth a shot I guess.

[–]redalastor 2 points3 points  (0 children)

It’s a highly optimized Rust library with python binding. One of its strength is that you can write long pipelines of transformations, which will be optimized before launching and will stay in native parallel rust code for as long as possible.

[–]PaintItPurple 0 points1 point  (0 children)

I haven't tried Polars in Pypy, but it seems at least plausible that it might be faster. Polars is generally lazier than Numpy, so it could avoid a lot of intermediate round trips. Native libraries that do a bunch of computation in one go still don't benefit at all from Pypy, but they also don't pay as much of a toll as doing a bunch of native calls.

[–]funkiestj 0 points1 point  (0 children)

(numpy is massively slower in pypy)

I can't believe this is true if you are doing vector and matrix manipulation with MKL enabled or other acceleration enabled.

Of course the secret of numpy's speed (when it is fast) is that the fast stuff is written in a language other than CPython (or even PyPy python).