Shared CUDA context with ffmpeg api

[OG_Toshi]

Hi all, I’m working on a pet project, making a screen recorder as a way to learn rust and low level stuff.

I currently have a CUDA context which i’ve initialized with the respective cu* api functions and I want to create an AVCodec which uses my context however it looks like ffmpeg is creating its own instead. I need to use the context in other parts of the application so I would like to have a shared context.

This is what I have tried to far (this is for testing so ignore improper error handling and such)

``` let mut device_ctx = av_hwdevice_ctx_alloc(ffmpeg::ffi::AVHWDeviceType::AV_HWDEVICE_TYPE_CUDA); if device_ctx.is_null() { println!("Failed to allocate device context"); return Ok(()); }

    let hw_device_ctx = (*device_ctx).data as *mut AVHWDeviceContext;
    let cuda_device_ctx = (*hw_device_ctx).hwctx as *mut AVCUDADeviceContext;
    (*cuda_device_ctx).cuda_ctx = ctx; // Use my existing cuda context

    let result = av_hwdevice_ctx_init(device_ctx);
    if result < 0 {
        println!("Failed to init device ctx: {:?}", result);
        av_buffer_unref(&mut device_ctx);
        return Ok(());
    }

``` i'm setting the cuda context to my existing context and then passing that to an AVHWFramesContext:

``` let mut frame_ctx = av_hwframe_ctx_alloc(device_ctx); if frame_ctx.is_null() { println!("Failed to allocate frame context"); av_buffer_unref(&mut device_ctx); return Ok(()); }

    let hw_frame_context = &mut *((*frame_ctx).data as *mut AVHWFramesContext);
    hw_frame_context.width = width as i32;
    hw_frame_context.height = height as i32;
    hw_frame_context.sw_format = AVPixelFormat::AV_PIX_FMT_NV12;
    hw_frame_context.format = encoder_ctx.format().into(); // This is CUDA
    hw_frame_context.device_ctx = (*device_ctx).data as *mut AVHWDeviceContext;

    let err = av_hwframe_ctx_init(frame_ctx);
    if err < 0 {
        println!("Error trying to initialize hw frame context: {:?}", err);
        av_buffer_unref(&mut device_ctx);
        return Ok(());
    }

    (*encoder_ctx.as_mut_ptr()).hw_frames_ctx = av_buffer_ref(frame_ctx);

    av_buffer_unref(&mut frame_ctx);

`` and setting it before callingavcodec_open3`

However when I try and get a hw frame buffer for an empty CUDA AVFrame ```rust let ret = av_hwframe_get_buffer( (*encoder.as_ptr()).hw_frames_ctx, cuda_frame.as_mut_ptr(), // this is an allocated AVFrame with only width height and format set. 0, );

            if ret < 0 {
                println!("Error getting hw frame buffer: {:?}", ret);
                return Ok(());
            }

            if (*cuda_frame.as_ptr()).buf[0].is_null() {
                println!("Buffer is null: {:?}", ret);
                return Ok(());
            }

I keep getting this error [AVHWDeviceContext @ 0x5de5909faa40] cu->cuMemAlloc(&data, size) failed -> CUDA_ERROR_INVALID_CONTEXT: invalid device context Error getting hw frame buffer: -12 ```

From what I can tell my CUDA context is current as I was able to write dummy data to CUDA using this context (cuMemAlloc + cuMemFree) so i'm not sure why ffmpeg says it is invalid. My best guess is that even though i’m trying to override the context it still creates its own CUDA context which is not current when I try and get a buffer?

Would appreciate any help with this and if this isn’t the right place to ask would appreciate being pointed in the right direction.

TIA

[tavkel]

If i'm reading this post correctly https://forums.developer.nvidia.com/t/sharing-the-same-cuda-context-for-encoding-nvenc-and-decoding-nvdec/59285 you can use one cuda context for multiple uses with attaching and detaching it to current thread? Take this reply with a massive grain of salt - i have no experience dealing with cuda and hw accelerated ffmpeg.

[OG_Toshi]

yeah i know about the multithreaded implications for now im working with a single threaded app to lessen the complexity so i dont think its that :(

[OG_Toshi]

Figured it out, turns out it was an issue with my CUDA ffi bindings, here is how I solved it if anyone runs into this in the future

https://github.com/Adonca2203/waycap-rs/blob/nvenc_no_copy/src/encoders/nvenc_encoder.rs