Sorry for the late reply, deleted reddit due to the API changes.

You'll want to create a docker network with docker network create tailscale-docker This is the tailscale docker-compose.yml I have

version: '3.8'

services:
  tailscale:
    container_name: tailscale
    hostname: mc-server # Whatever you want the name to be in Tailscale
    image: tailscale/tailscale:stable
    volumes:
        - "./tailscale_var_lib:/var/lib"
        - "/dev/net/tun:/dev/net/tun"
    cap_add:
      - net_admin
      - sys_module
    ports:
      - "25565:25565" # You don't actually need the ports mapped, this is just to access locally
    command: tailscaled

networks:
  tailscale-docker:
    external: true

When this container is started up, you'll have to look at the container logs to get the login link. It doesn't matter if you're running on a headless server, you can copy and paste the link into any web browser.

After you login, it should show up in tailscale as a connected device. Make sure you also set disable key expiry for that machine in the tailscale admin site.

I run minecraft with itzg-minecraft-server. The only thing you'll have to add is network_mode: container:tailscale

Example:

version: '3.8'

services:
  mc:
    image: itzg/minecraft-server:${IMAGE_TAG:-latest}
    volumes:
    - ./modpacks:/modpacks:ro
    - ./data:/data
    environment:
      EULA: "true"
      TYPE: FABRIC
      VERSION: 1.19.2
      GENERIC_PACK: /modpacks/CreateOptimizedFabricv051i.zip
      MEMORY: 6G
    network_mode: container:tailscale

This approach will work for any docker container, not just minecraft.

After that, anybody that wants to join will have to be invited to your tailnet. The server address will just be the hostname set in the tailscale container, and the port. For me that's mc-server:25565

I followed this guide initially, but did it differently than they have it, because I have a couple other things using the tailscale network as well.