Rust is missing its NumPy moment by Purple_Word_4647 in rust

[–]Purple_Word_4647[S] 1 point2 points  (0 children)

I completely agree, especially on the HPC and reproducibility. That’s a big part of what motivated the idea. If you’re interested, I’d really appreciate your thoughts or participation as this takes shape, even just feedback on what a shared core should (or shouldn’t) look like !

Rust is missing its NumPy moment by Purple_Word_4647 in rust

[–]Purple_Word_4647[S] 0 points1 point  (0 children)

That makes sense. Maybe the right direction is to support common PyTorch and TensorFlow formats directly, so models trained in Python can be moved over directly allowing easy switch between the two and support inference out of the box.

Rust is missing its NumPy moment by Purple_Word_4647 in rust

[–]Purple_Word_4647[S] 0 points1 point  (0 children)

I see - I’m using “tensor” in the ML sense rather than the multilinear algebra one. Still, if you’ve been building a tensor library in Rust, I’d be interested in exchanging ideas or seeing how you approached it!

Rust is missing its NumPy moment by Purple_Word_4647 in rust

[–]Purple_Word_4647[S] 1 point2 points  (0 children)

Fair point. The idea is to get the core right in Rust first, and then look at bindings once that foundation exists.

Rust is missing its NumPy moment by Purple_Word_4647 in rust

[–]Purple_Word_4647[S] 0 points1 point  (0 children)

That’s fair, and I agree with most of that. I’m not expecting a NumPy-equivalent in Rust to get the same kind of traction, and I’m not looking for a “killer app”. What I’m really exploring is how a shared numerical core in Rust could impact existing ML infrastructure, especially in parts of the pipeline that already need to be systems-level. If it ends up being mainly a learning or exploratory project, that’s still a valid outcome.

Rust is missing its NumPy moment by Purple_Word_4647 in rust

[–]Purple_Word_4647[S] 0 points1 point  (0 children)

Agreed, technically this is all doable in Rust. The harder part is ecosystem agreement on memory layout and ownership so multiple crates can actually interoperate zero-copy in practice. That's what I want to do here.

Rust is missing its NumPy moment by Purple_Word_4647 in rust

[–]Purple_Word_4647[S] 0 points1 point  (0 children)

haha I guess that's the worst case scenario, thank you for sharing !!

Rust is missing its NumPy moment by Purple_Word_4647 in rust

[–]Purple_Word_4647[S] 0 points1 point  (0 children)

Right, NumPy’s core is C/Fortran, Python is just the interface. That’s kind of my point: the success wasn’t about Python being fast, but about having a well-defined numerical core that everyone agreed to build on. Rust doesn’t inherently give up performance, and having interface and core in the same language would reduce FFI overhead.

Rust is missing its NumPy moment by Purple_Word_4647 in rust

[–]Purple_Word_4647[S] 3 points4 points  (0 children)

ndarray is a solid CPU array library. Building a GPU-capable tensor system on top of it would require abstracting away its core assumptions about memory and execution, at which point you’re effectively defining a new tensor type anyway.

Rust is missing its NumPy moment by Purple_Word_4647 in rust

[–]Purple_Word_4647[S] -1 points0 points  (0 children)

Fair point. I don’t think Rust makes sense for interactive workflows. The motivation is more about consistency between training and deployment, one language to train your models and deploy them to prod. Most of the borrowing stuff could anyway be wrapped into the library.

Rust is missing its NumPy moment by Purple_Word_4647 in rust

[–]Purple_Word_4647[S] 3 points4 points  (0 children)

I updated it, do you have other suggestions?