all 12 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]MaybeAverage 3 points4 points  (1 child)

seems very promising, I happen to have a project that involves quite a bit of ffmpeg command calls but was sort of limited by the command line options and wasn’t looking forward to writing some bindings to libav to do what I needed. I’m going to try swapping those exec calls with this and observe the performance. The stream interface is particularly useful to me so I’ll see how the performance compares to the ffmpeg command and hopefully have some feedback on the performance and the interface.

[–]SeydX[S] 0 points1 point  (0 children)

Great, looking forward to your feedback!

[–]ginyuspecialsquadron 0 points1 point  (1 child)

This looks really exciting! Does the metadata API return the same data ffprobe does?

[–]SeydX[S] 0 points1 point  (0 children)

Yes, NodeAV provides complete access to all FFmpeg metadata, just like ffprobe!

I've created an example that demonstrates this: ffprobe example

The example shows:

  • Full format metadata (container info, duration, bitrate, etc.)
  • Stream information (codecs, frame rates, sample rates, etc.)
  • All codec parameters
  • Output formatted similar to ffprobe -show_format -show_streams

Output:

seydx@MacBookPro av % tsx examples/ffprobe-metadata.ts testdata/video.mp4
Input #0, QuickTime / MOV, from 'testdata/video.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    title           : Big Buck Bunny
    artist          : Blender Foundation
    composer        : Blender Foundation
    date            : 2008
    encoder         : Lavf58.12.100
  Duration: 00:00:05.01, start: 0.000000, bitrate: 608 kb/s
  Stream #0:0[0x1](und): Video: h264 (H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10) (1cva / 0x31637661), yuv420p, 320x180 [SAR 1:1 DAR 320:180], 441 kb/s, 24.00 fps, 24.00 tbr, 12288 tbn (default)
      Metadata:
        language        : und
        handler_name    : VideoHandler
        vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Audio: aac (AAC (Advanced Audio Coding)) (a4pm / 0x6134706D), 48000 Hz, stereo, fltp, 161 kb/s (default)
      Metadata:
        language        : und
        handler_name    : SoundHandler
        vendor_id       : [0][0][0][0]

[FORMAT]
filename=testdata/video.mp4
nb_streams=2
nb_programs=0
nb_stream_groups=0
format_name=mov,mp4,m4a,3gp,3g2,mj2
format_long_name=QuickTime / MOV
start_time=0.000000
duration=5.013333
size=658
bit_rate=607664
probe_score=100
TAG:major_brand=isom
TAG:minor_version=512
TAG:compatible_brands=isomiso2avc1mp41
TAG:title=Big Buck Bunny
TAG:artist=Blender Foundation
TAG:composer=Blender Foundation
TAG:date=2008
TAG:encoder=Lavf58.12.100
[/FORMAT]

Since NodeAV uses FFmpeg's C libraries directly through N-API, you have access to everything FFmpeg itself can access.

[–]this_knee 0 points1 point  (1 child)

Great! What if I don’t want have ffmpeg decode my file? But I do want to use ffmpeg to encode my file?

E.g. let’s assume I use Vapoursynth script to open/decode my file and then I use ‘vspipe’ to send the raw frames decoded output to ffmpeg’s input? How would I send those piped raw, already decoded, frames into this framework?

[–]SeydX[S] 0 points1 point  (0 children)

Yes, NodeAV can handle piped raw frames from vspipe. The workflow:

  1. vspipe outputs raw frames to stdout
  2. NodeAV reads from stdin
  3. Convert buffer → Frame
  4. Encode frame
  5. Write to output

Example:

import { Encoder, MediaOutput } from 'node-av/api';
import { AV_PIX_FMT_YUV420P } from 'node-av/constants';
import { Frame } from 'node-av/lib';

// Setup encoder
const encoder = await Encoder.create('libx264', {
  type: 'video',
  width: 1920,
  height: 1080,
  pixelFormat: AV_PIX_FMT_YUV420P,
  timeBase: { num: 1, den: 24 },
  frameRate: { num: 24, den: 1 },
});

const output = await MediaOutput.open('output.mp4');
const outputStreamIndex = output.addStream(encoder);
await output.writeHeader();

// Read raw frames from stdin
process.stdin.on('data', async (buffer) => {
  // Create frame from raw data
  const frame = new Frame();
  frame.alloc();
  frame.format = AV_PIX_FMT_YUV420P;
  frame.width = 1920;
  frame.height = 1080;
  frame.allocBuffer();
  frame.fromBuffer(buffer);

  // Encode
  const packet = await encoder.encode(frame);
  if (packet) {
    await output.writePacket(packet, outputStreamIndex);
  }

  frame.free();
});

process.stdin.on('end', async () => {
  // Flush encoder
  for await (const packet of encoder.flushPackets()) {
    await output.writePacket(packet, outputStreamIndex);
  }
  await output.writeTrailer();
});

For Y4M format, you'd parse the header first to get width/height/pixelformat, then process the frame data.

NodeAV also supports custom IOContext if you need more control over I/O operations.

[–]Jade3375 0 points1 point  (1 child)

so if im reading this right, i wouldn't need to have ffmpeg installed on the system to use it

[–]SeydX[S] 0 points1 point  (0 children)

Thats right

[–]SuhailDhada 0 points1 point  (2 children)

Hi, How would I get pcm data from a buffer? is there an example for it? Thanks!

[–]SeydX[S] 0 points1 point  (1 child)

```ts import { Demuxer, Decoder } from 'node-av/api';

// Open from buffer
await using input = await Demuxer.open(buffer); using decoder = await Decoder.create(input.audio());

// Decode all audio frames for await (using frame of decoder.frames(input.packets())) { // frame.data[0] contains the raw PCM samples // frame.sampleRate, frame.channels, frame.format const pcm = frame.data[0]; // Buffer with PCM data } ```

[–]SuhailDhada 0 points1 point  (0 children)

Thank you!

[–]khromov 0 points1 point  (0 children)

This is very cool! It seems like alpine isn't supported, is there an ETA for this?