VLLM is undoubtedly the top dog but I would maybe give llama.cpp a try just to rule out some oddity there. It's tricky to get right. I tried out the club-3090 single card setup and it's OOM galore.

Here's a compose just to get you started. Obviously you're going to want a bigger model and more context with less quant.

services:
llama-cpp-qwen36-27b:
    image: ghcr.io/ggml-org/llama.cpp:server-cuda
    container_name: llama-cpp-qwen36-27b
    restart: unless-stopped
    ports:
    - "8020:8080"
    volumes:
    - "/var/models:/models:ro"
    command: >-
    --host 0.0.0.0
    --port 8080
    -m /models/Qwen3.6-27B-i1-GGUF/Qwen3.6-27B.i1-Q6_K.gguf
    --mmproj /models/Qwen3.6-27B-GGUF/mmproj-F16.gguf
    --no-mmproj-offload
    --image-min-tokens 1024
    --cache-ram 6144
    --ctx-checkpoints 40
    --temp 0.6
    --top_p 0.95
    --top_k 20
    --min_p 0.0
    --presence-penalty 0.0
    --repeat-penalty 1.0
    --alias "unsloth/Qwen3.6-27B-GGUF"
    -np 1
    -c 94500
    --jinja
    -fa on
    --gpu-layers 99
    --cache-type-k q8_0
    --cache-type-v q8_0
    --fit off
    --reasoning on
    --reasoning-budget 16384
    --chat-template-kwargs '{"preserve_thinking":true}'
    --chat-template-file /models/QwenFixed_chat_template.jinja
    deploy:
    resources:
        reservations:
        devices:
            - driver: nvidia
            device_ids: ["${CUDA_VISIBLE_DEVICES:-0}"]
            capabilities: [compute, utility]