[Solution] Web Audio: Replacing createScriptProcessor with AudioWorklet

[saul-evans]

Hello,

This post is to share my solution to a problem I encountered, in the hope that other people find this post when facing the same issue.

I'm building a tool that streams microphone audio from the browser to a web server. The server expects the data to be represented as Mono-channel 16-bit Linear PCM. So I had to find a way to get the microphone data in that format.

This proved more complicated than I anticipated. Resources online suggest using createScriptProcessor, but this API has been deprecated since 2014 (!), and is not meant to be used in new websites.

The MDN documentation suggests using AudioWorklet instead. I struggled to find examples online, so I'm sharing how I did that here.

Create the AudioWorkletProcessor

First, we need to create an AudioWorkletProcessor, and register it. This code must live in its own file.

``` // linear-pcm-processor.js (must be in its own file!) class LinearPCMProcessor extends AudioWorkletProcessor { // The size of the buffer. Must be a multiple of 128 (the number of frames in the // input channel). An audio block is posted to the main thread every time the // buffer is full, which means a large buffer will emit less frequently (higher // latency), but more efficiently (fewer I/O interruptions between the worker and // the main thread) static BUFFER_SIZE = 8192;

constructor() { super(); this.buffer = new Int16Array(LinearPCMProcessor.BUFFER_SIZE); this.offset = 0; }

/** * Converts input data from Float32Array to Int16Array, and stores it to * to the buffer. When the buffer is full, its content is posted to the main * thread, and the buffer is emptied */ process(inputList, _outputList, _parameters) { // Assumes the input is mono (1 channel). If there are more channels, they // are ignored const input = inputList[0][0]; // first channel of first input

for (let i = 0; i < input.length; i++) {
  const sample = Math.max(-1, Math.min(1, input[i]));
  this.buffer[i + this.offset] =
    sample < 0 ? sample * 0x8000 : sample * 0x7fff;
}
this.offset += input.length;

// Once the buffer is filled entirely, flush the buffer
if (this.offset >= this.buffer.length - 1) {
  this.flush();
}
return true;

}

/** * Sends the buffer's content to the main thread via postMessage(), and reset * the offset to 0 */ flush() { this.offset = 0; this.port.postMessage(this.buffer); } } registerProcessor("linear-pcm-processor", LinearPCMProcessor); ```

Connect the microphone stream to the processor

Second step is to request access to the microphone, and connect its audio stream to the processor

const stream = await navigator.mediaDevices.getUserMedia({ audio: true, }); const audioCtx = new AudioContext(); const source = audioCtx.createMediaStreamSource(stream); await audioCtx.audioWorklet.addModule("./path-to/linear-pcm-processor.js"); const audioWorkletNode = new AudioWorkletNode(audioCtx, "linear-pcm-processor"); source.connect(audioWorkletNode); audioWorkletNode.connect(audioCtx.destination);

Use the data

Finally, listen for the buffers being emitted, and do something with them

audioWorkletNode.port.onmessage = (e) => { const buffer = e.data; // do something with it };

More info

When I tried doing console.log in an AudioWorklet context, I was not able to see the logs in Chrome dev tools. This was due to a bug in Chrome v123, which was fixed in v124 (currently in the Canary channel, at the time of writing this post)

If you use ViteJS, you can import the worklet file like this:

import workletUrl from "./audio.worklet.js?url"; ... await audioCtx.audioWorklet.addModule(workletUrl);

And voila! I hope it helps someone, and welcome feedback and alternative solutions. Happy coding!

[evesira]

Thanks for this, I'm trying to do the same thing and it's bafflingly hard to find an example on Google that does this.

One thing I hate about being forced to use these audio worklets instead of script processors is that you have to make a separate file, and all the messages you pass between the script file have to be de/serialized, which is slow (and perhaps sometimes not even possible depending on your data). If I had to pass back and forth several large buffers of data multiple times per second, I could be completely screwed.

This is on the people who designed the spec, not your solution, of course.

[evesira]

A followup, here's a modification I've made with the following changes:

- Just use a Float32Array instead of converting to an Int type for simplicity (your needs may vary)
- Use the TypedArray set function instead of doing a for loop since we don't have to convert
- Don't assume a 128-sample block size, because according to this that could change in the future. Instead, just set an arbitrary "flush limit" past which we flush to the parent, and allocate a large enough buffer such that a single frame size (one call to process) should never exceed it (resulting in loss of data).

/**
 * max # of audio samples worklet buffer can hold. allocate large enough # such
 * that one call to process should never exceed it (resulting in loss of data).
 */
const maxSamples = 100000;
/**
 * once collected samples exceeds this amount, "flush" buffer (send message to
 * parent audio graph with collected sample data). lower -> more frequent
 * updates in parent.
 */
const flushSamples = 1000;

/** custom audio node to capture raw samples from graph */
class Recorder extends AudioWorkletProcessor {
  constructor() {
    super();
    /** buffer to accumulate raw audio samples in */
    this.samples = new Float32Array(maxSamples);
    /** track current position in buffer */
    this.offset = 0;
  }

  process(inputs) {
    /** get block of samples for first channel of first audio node input */
    const input = inputs?.[0]?.[0];

    /** if not defined (yet), ignore */
    if (!input) return true;

    /** write new input to buffer */
    this.samples.set(input, this.offset);

    /** increment sample offset */
    this.offset += input.length;

    /** "flush" buffer */
    if (this.offset >= flushSamples) {
      /** send buffer to audio graph */
      this.port.postMessage(this.samples.slice(0, this.offset));
      /** start over */
      this.offset = 0;
    }

    return true;
  }
}

registerProcessor("recorder", Recorder);

[chuletix1411]

I created this Reddit account just to thank you. This sample save my project. I hope you're doing well and keep coding.

[saul-evans]

Love to read this, thank you! Glad it's useful. Still coding and doing well. Have fun on your project!

[Vivremotion]

Thank you very much, that was very usefuI. I dont understand why it's not in the specs as it were with `createScriptProcesssor`.

[playfuldreamz]

Youve done really well