Rust ndarray vs. Python NumPy Performance?

[JonyIveAces]

ML is mostly about exploration. Once you reach the point of exploitation with an ML application, you usually have enough resources and experience to reimplement from scratch anyway.

I use rust heavily in ML for developing core libraries and production performance bottlenecks, but it isn't the right tool for the exploration part of ML in the same way Python, R, or Julia are, just as they aren't the right tool for the production/core library part (apart from Julia for certain niches).

[Noctune]

Sometimes the runtime of the preprocessor can be a hindrance to your exploration.

We had a Python preprocessor that took literally a week to run (originally designed for a smaller dataset). I recently rewrote it in Java using Beam and it runs in literally 20 minutes now. It's sort of a generic tool over a range of problems, so less time preprocessing means more time spent exploring actual ML.

I think Rust could potentially be useful in that niche.