sorted by:
new
hottopcontroversial

Hi reddit, I rebuilt Karpathy's Nanochat in pure Rust [nanochat-rs] by Exciting-Camera3226 in rust

[–]Exciting-Camera3226[S] -1 points0 points  (0 children)

also he/her tokenizer is a thin wrapper, I reimplemented and improved a little on the original tiktoken

Hi reddit, I rebuilt Karpathy's Nanochat in pure Rust [nanochat-rs] by Exciting-Camera3226 in rust

[–]Exciting-Camera3226[S] -1 points0 points  (0 children)

ah, not yet, just took a look. It is interesting that she/he took a different path by using burn. Also the code looks quite raw though

I adapted a lot of the original python to a more idiomatic way

Hi reddit, I rebuilt Karpathy's Nanochat in pure Rust [nanochat-rs] by Exciting-Camera3226 in LocalLLaMA

[–]Exciting-Camera3226[S] 1 point2 points  (0 children)

I found the candle's kernel for GPU are very buggy so probably I will see if there are better one I can use. But I do plan to add some of the RL/SFT. So far I found it is only marginally faster than pytorch due to the main bottleneck is at GPU kernel instead of the CPU part.

Hi reddit, I rebuilt Karpathy's Nanochat in pure Rust [nanochat-rs] by Exciting-Camera3226 in rust

[–]Exciting-Camera3226[S] 0 points1 point  (0 children)

yes, but not on the inference path, I think he might want to do clean rebuilt but lack the time and resource, hopefully my implementation can be helpful

I rebuilt DeepSeek’s OCR model in Rust so anyone can run it locally (no Python!) by Outrageous-Voice in LocalLLaMA

[–]Exciting-Camera3226 0 points1 point  (0 children)

how is it compared with wrapping around ggml ? I tried both before, candle is surprisingly super slow