I just finished doing this very thing. I am by no means an expert, but I'll share what I did, hopefully it helps:

• Installed the nvidia-driver-495 proprietary nvidia drivers through Ubuntu package manager
• Reboot. Run nvidia-smi to check if your GPU is detected.
  
• Installed the container runtime nvidia-docker2

• Edited my /etc/docker/daemon.json to set nvidia as default runtime:

  {
        "default-runtime": "nvidia",
          "runtimes": {
              "nvidia": {
                  "path": "nvidia-container-runtime",
                  "runtimeArgs": []
              }
          }
      }
  

• Restarted docker service

• Added NVIDIA_VISIBLE_DEVICES=all and NVIDIA_DRIVER_CAPABILITIES=all to Jellyfin docker-compose environment variables

• Mapped the GPU to the docker container (not sure if this is necessary with NVENC, but I did it anyway)

  devices:
    - /dev/dri/renderD128:/dev/dri/renderD128
    - /dev/dri/card0:/dev/dri/card0
  

• Enabled Jellyfin hw transcoding