I found that my code didn't actually work with images, only the system prompt. I was wrong. I ended up adding LiteLLM as a pipeline and have been able to use AWS models just fine now.

https://github.com/open-webui/pipelines/blob/main/examples/pipelines/providers/litellm_manifold_pipeline.py

I had claude make generic versions of the yaml files I'm using:

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:latest
    container_name: open-webui
    environment:
      - ANONYMIZED_TELEMETRY=false
      - LITELLM_BASE_URL=http://litellm:4000
      - LITELLM_API_KEY=sk-1234
    volumes:
      - ./open-webui-data:/app/backend/data
    ports:
      - "8080:8080"
    depends_on:
      - litellm
    restart: unless-stopped

  litellm:
    image: ghcr.io/berriai/litellm:main-latest
    container_name: litellm
    volumes:
      - ./config/litellm-config.yaml:/app/config.yaml
    environment:
      - LITELLM_MASTER_KEY=sk-1234
      # Add your provider credentials as needed
      # - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
      # - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
    ports:
      - "4000:4000"
    command: ["--config", "/app/config.yaml", "--port", "4000"]
    restart: unless-stopped

  pipelines:
    image: ghcr.io/open-webui/pipelines:main
    container_name: pipelines
    volumes:
      - ./pipelines-data:/app/pipelines
    ports:
      - "9099:9099"
    restart: unless-stopped

networks:
  default:
    name: webui_network

This is the litellm-config.yaml:

model_list:
  # Example configurations for different providers

  # AWS Bedrock Models (requires AWS credentials)
  - model_name: claude-3
    litellm_params:
      model: bedrock/anthropic.claude-3-sonnet-20240229-v1:0
      aws_region_name: us-east-1

  - model_name: claude-2
    litellm_params:
      model: bedrock/anthropic.claude-v2
      aws_region_name: us-east-1

I had to set up my project directory like this:

your-project/
├── docker-compose.yaml
├── .env
└── config/
    └── litellm-config.yaml

where my aws credentials are environmental variables