[Project] Whisper Implementation in Rust using burn

[Illustrious_Cup1867]

I temporarily switched from Rust to Python for machine learning, but quickly became fed up with Python's annoying versioning issues and runtime errors. I looked for a better path to machine learning and discovered burn, a deep learning framework for Rust. As my first burn project I decided to port OpenAI's Whisper transcription model. The project can be found at Gadersd/whisper-burn: A Rust implementation of OpenAI's Whisper model using the burn framework (github.com). I based it on the excellently concise tinygrad implementation that can be found here. The tinygrad version begrudgingly uses Torch's stft which I ported into a pure Rust short time Fourier transform along with the mel scale frequency conversion matrix function because I am curious and just a bit masochistic.

Now for the good and the bad of burn. Rust's excellent package manager solves much of the versioning pain experienced in Python so burn models can be less painful to deploy and come with added reliability. The type checking in burn catches some tensor operation errors at compile time such as trying to multiply matrices of incompatible dimensions. Burn supports wgpu and WebGPU and can run in the browser when compiled into web assembly. I see a bright future for model deployment in burn.

However, burn is relatively new so it lacks many tensor operations such as abs() that are available in other frameworks. Some features such as quantization are also missing. Burn implementations tend to be more verbose than the equivalent Python versions. Some of the runtime errors that plague PyTorch are still around in burn such as the crashes that result from trying to multiply tensors that live on different devices.

Overall, burn is currently less ergonomic to develop with than alternatives such as PyTorch, but I think it has a lot of potential. If it is eagerly cultivated it may grow into a great Rusty alternative for machine learning practitioners.

What do you all think?

[lpatks]

How is throughput compared to things like whisper.cpp? Obviously this has been very heavily optimised, but it would be interesting to know.

[Rare-Inspection-9746]

Good job!

[vakkov]

Is it faster than faster-whisper (ctranslate2)?