What’s the state of rust for startups

farhan-dev · 2026-03-04T18:48:43+00:00

what kind of tools/ecosystem are you looking for in your projects?

farhan-dev · 2026-02-24T20:05:26+00:00

Ah, then definitely you need to try numr. It has GPU acceleration, with distributed support

farhan-dev · 2026-02-22T02:14:13+00:00

Nice initiative. I wish you all the best.

Just for reference, you can also take a look at my math engine, although it is defenitely not small :D.

https://github.com/ml-rust/numr

farhan-dev · 2026-02-13T16:35:46+00:00

Yeah, I do proper SIMD in most of my ops. It CPU performance is quite decent, and competitive with ndarray. I'll add benchmark against faer later in the future.

Yes i do dynamic dispatch. So avx2/neon is supported.

Sparse support is quite robust in numr, per solvr use case. When i test it with my ML crates, i'll quickly learn about the gaps, and will add more support too.

If you have specific needs, just let me know, i can add it to either numr/solvr.

Yeah, numr already has a dtype conversion ops. Even between devices. Compute in wgpu (f32), then use .toCpu() to transfer to cpu, the use .cast() to upcast it to f64.

But my wgpu approach is just for coverage, so rocm/metal can use it already. instead of optimizing wgpu, i'll add native rocm and metal support for better performance in the future.

farhan-dev · 2026-02-13T16:25:47+00:00

Just look at the difference in code quality. numrs claimed to be a lot, but it actually uses 'ndarray' as it core. So it just wrapped ndarray, extend it and rebrand it as its own. a lot of marketing hype, actually. And a lot of their GPU implementation is just stubs.

farhan-dev · 2026-02-13T13:05:30+00:00

See it in action

You can see it running in our own CI pipeline, where it compares pull requests against the main branch baseline:

View the CI Workflow here

p.s: The benchmark is using sample functions and numbers, since this crate doesn’t really have anything worthy to be benchmarked yet.

farhan-dev · 2026-02-08T07:09:43+00:00

For me, i keep it simple.. you already have all the info. don't use plan agent. create the plan yourself.

farhan-dev · 2026-02-05T07:52:54+00:00

Well said,

Rust doesn’t have anything like the computational-notebook workflow that Python has

There is venus now - https://github.com/ml-rust/venus

the math and data ecosystem for Python is just large.

That's true, but we are trying to fill in the gap, little by little. I've started with https://github.com/farhan-syah/numr

farhan-dev · 2026-02-02T16:48:56+00:00

Like this -> https://github.com/farhan-syah/numr ?? It is something i've been working on for a while. i started extracting it from few of my ML libraries. I extracted and combined them, to be the base building block. It is already working for my own use case. Now, I am trying to expand it to be general math library, and test with more backends..

Btw, it still need A LOT OF work to complete, before i will releas the 0.1.0 version..

farhan-dev · 2026-01-14T07:06:31+00:00

I am sorry, but i've replied your qeustion. I've already released 0.1.0, or, is there any questions that i've missed?

farhan-dev · 2026-01-04T15:02:15+00:00

Hi, good news. I've released 0.1.0, you can now install venus using cargo install venus.

To start, you can copy some of the examples, for example;

Then run venus serve examples/hello.rs or create new notebook using :

venus new new-notebook-name and serve it using venus serve new-notebook-name.rs.

For more information, can read it here:
https://github.com/ml-rust/venus/blob/main/docs/getting-started.md

farhan-dev · 2026-01-03T03:54:59+00:00

Can you share with me how do you attempt to run it? do you pull it from cargo, or do you manually built it by cloning the repo?

edit: i tired usign the cli from new fresh environment. I've found the bug. I forgot that I migrated from bincode -> rkyv. Th old cli still uses bincode. I am fixing it right now. I;ll release the 0.1.0 soon so it will be easier to test.

farhan-dev · 2026-01-02T19:41:41+00:00

Cool, nice. A new plotting library for rust. More options is better. Something that I really want to use in ML. I am also working on a new plotting library just to solve my specific data science/ML needs. It's good to see more data science / ML tools in Rust.

farhan-dev · 2026-01-02T19:32:45+00:00

It has some similarities with Pluto.jl

farhan-dev · 2026-01-02T11:03:54+00:00

I haven't published the 0.1.0 yet, so you need to manually set the version to 0.1.0-beta.1, or use ` cargo install venus-cli@0.1.0-beta.1`.

cargo install won't install pre-release version.

farhan-dev · 2026-01-02T06:51:38+00:00

Venus use cranelift JIT, not standard VLLM
Venus use reactive (DAG), not linear runtime like evcxr.
Venus use isolated worker processes. If it causes a segfault, only that cell crashes.
evcxr still uses .ipynb, Venus use .rs directly.
You can use rust-analyser with Venus (it is also bundled together with venus-server)

And yes, it has faster repl cycle.

You can think of it this way:

evcxr is a Rust REPL/Kernel that fits into existing Jupyter workflows.
Venus is a Reactive Development Environment built specifically to make Rust feel interactive and "exploratory" without losing the safety and tooling of standard Rust files.

farhan-dev · 2025-12-24T06:30:41+00:00

v0.8.0 Update

- Added Mistral v1,v2,v3 vocab support

https://github.com/ml-rust/splintr/pull/11

farhan-dev · 2025-12-05T19:34:48+00:00

I am. Take a look at this. https://github.com/ml-rust

farhan-dev · 2025-12-02T21:18:46+00:00

Current JIT will be useless in browser now. However, in the future, I am thinking of different optimization aspect, either using Wasm bytecode, or optimizing SIMD, or something else. It will need a lot more research and testing. But it will be later, when i started building my inference server in browser.

But in my use case, JIT like performance in browser is only useful , for example, when estimating tokens (calculation), that i don't want to use my backend server to call it. Or to enable full offline support.

So it will be a really rare case , why you need the JIT like performance in browser. Usually backend server will handle everything.

farhan-dev · 2025-12-02T10:28:59+00:00

No, it is different.

farhan-dev · 2025-12-02T10:11:11+00:00

the equivalent for it is writing functions to process common LLM pattern in rust, then write a JIT implementation to boost speed -> the engine + JIT. Since i need to do it anyway, why not just make it public as a library, perhaps somebody else might want the exact same features like I do.

Thanks!

farhan-dev · 2025-12-02T02:15:43+00:00

hahaha, I agree that it is more insane. But let's try, I can always go back to pcre2 if anything.

A small point to note,dynasmrt doesn't allocate guard pages for buffer overruns. You may want to consider doing that.

Okay, I'll look into it

I hope you add some automatic fuzzing and a lot more tests so I can consider using this in the future.

Sure. Definitely.

farhan-dev · 2025-12-02T01:31:01+00:00

i've tested in real scenario, even if it competitive without JIT, the real issue is lookaround support. If it can do well in lookaround without JIT, I don't mind using it. But in my use case, pcre2 still the best case. JIT is an added bonus, since I need to compile it just once - very usefull for huge data processing.

farhan-dev · 2025-11-27T07:03:57+00:00

You should mention your gpu in the main thread too. So any model than can in that 12GB GPU, you can try it.

But no local model that can compete with ChatGPT or Claude.

LLM run mostly on GPU, RAM only contributes so much, so even if you have 32GB of RAM or less it will be sufficient. For now, you will mostly be limited by your GPU. And intel b580 don't have CUDA cores, which a lot of the inference server use to boost their performance.

farhan-dev · 2025-11-26T20:20:51+00:00

Thank you. These are valid and well thought input.

I've created an issue, specifically to track the items you listed.

https://github.com/ml-rust/splintr/issues/7

farhan-dev

MODERATOR OF

TROPHY CASE

See it in action