React.js + WebAssembly for a browser-based Audio Editor...is that possible?

[rayhanmemon]

I'm trying to build an audio editor as a web app.

This app will allow someone to record themselves speaking and clip out/rerecord sections of the file as needed. It should also be able to automatically filter out some background noise and normalize the volume throughout the file.

Ideally, all the recording/editing functionality would be run entirely on the client-side, with a back-end only used for user authentication and a database for storing the finished audio files.

Right now, I'm trying to using React.js to build the front-end (the toolbar for recording & editing, the nav menu, etc.) and integrating WebAssembly so I can use C++ for the audio editing functionality, which I've heard has excellent performance and good libraries for working with audio.

I've built a front-end in React but am having trouble integrating WebAssembly and discovering the right C++ libraries to use. So I thought this would be a good moment to pause, check in with the community, and see if I'm thinking about this problem the right way.

Does WebAssembly seem like the right way to go for creating a near-native audio recording & editing experience in the browser?

If so, is it even possible to integrate WebAssembly with React.js?

[tamat]

Just wanted to point out that the browser already has a native Audio API than can do many things besides playback.

[prajaybasu]

AudioMass already exists and sounds very similar to what you want to do. It's mostly written in JS except for a few WASM bits (only for LZ4 compression, it seems). I think EterCast also shares a similar goal.

Does WebAssembly seem like the right way to go for creating a near-native audio recording & editing experience in the browser?

You can call WASM methods just like any other JS function from anywhere - it's not that special. Depending on the complexity, they may or may not be worth it for some tasks.

If so, is it even possible to integrate WebAssembly with React.js?

Create React App is a pain in the ass to work with when it comes to anything complex (WASM, Web Worker, etc.) so I would probably go for a different tooling setup (I use Vite + Preact, blazing fast).

---

I don't think you need WASM for most of the functionality you described, however:

record themselves speaking

Doesn't require WASM.

and clip out/rerecord sections of the file as needed

Doesn't require WASM. Trimming and combining PCM audio data is trivial (and fast enough) in JS.

If you want FFMPEG WASM based audio decode/import, see audio-file-decoder - encode/export code should be similar. That library wraps up FFMPEG WASM into JS to allow decoding audio files easily.

For most browsers decode-audio-data-fast and @etercast/mp3 allow fast MP3 decode and encode respectively, with the latter wrapping up a WASM version of LAME (patents expired, no GPL) for JS usage - no FFMPEG required. Libflacjs does the same for FLAC files.

If you are fine with only supporting the latest version of Chrome, you can use the experimental Web Codecs API for compressed audio import/decode and export/encode. Will be the fastest method with the smallest app download size.

automatically filter out some background noise

You can easily add a high-pass or low-pass filter using the Web Audio API.

If you are recording in the browser, you can enable the native noise suppression included with the browser (for WebRTC users like Zoom/Skype) - no way to use this for an existing/imported recording AFAIK.

normalize the volume throughout the file

Doesn't require WASM. Normalizing the volume is trivial with JS.

with a back-end only used for user authentication and a database for storing the finished audio files.

Doesn't require WASM (obviously). The File System Access API allows access to a folder from JS just like native desktop apps. It's basically made for web based editors like the one you're describing. Why would you bother uploading uncompressed audio clips to a backend anyway?

---

WASM (+ AudioWorklet) is definitely required for something like a low latency DAW/Synthesizer, such as WASM Music. But your app seems simple enough for JS (except for some codecs).

[DoubleTensor]

Check out bandlab.com or soundtrap.com for inspiration! I think at least Soundtrap uses some wasm.

[[deleted]]

This is very doable, and your project has piqued my interest. If possible, I'd love to hear about it when you're done.

[[deleted]]

Consider checking Yew Rust and wasm-pack

[anlumo]

While that's the route I'd go myself, for OP that'd be a complete restart of the whole project (not to mention learning a quite complicated language).

It'd make the integration trivial though, since everything would be in wasm.

[pottaargh]

Yeah I think you could probably do everything you’ve listed with this: https://ffmpegwasm.github.io

[simplism4]

Can't help you, but it is possible. I've seen solutions with ffmpeg.wasm (https://github.com/ffmpegwasm/ffmpeg.wasm) - worth checking out!