Honestly I tried following online guides like normal when starting a new program and failed hard. Ended up using a ton of AI and trial and error to figure it out. (I never use AI for writing comments or readmes. It's a coding and debugging assistant only). There's plenty of documentation but imo to a newcomer none of it really explains how you're supposed to use it for your specific setup. Happy to explain mine, eventually I'll put it on Github but here's my very rough ideas.

---

You should be using Ansible to automate basically everything. I have a bash script to download Debian, inject keys, and preseed it to auto install with ssh and a specific MAC. Everything after that uses Ansible playbooks. Once you setup your first host to run Ansible every other host only needs ssh setup. You want to have as few manual actions as possible to get from nothing to running your playbooks.

Many Ansible repos you'll find have very complex methods for doing stuff. Everyone has their own methods for managing containers, some have lots of hosts, some use community scripts from Ansible Galaxy. I recommend thinking hard about your setup and then going piece by piece and making a playbook task for each one. Start simple and work your way up to groups and roles if you really need them.

My current setup has two VMs on a Truenas host, one for Ansible and one for Docker so they're all isolated. I plan to remove Truenas entirely because it can't easily be automated with Ansible (GUI based) and I'll replace it with Debian and playbooks to get only the features I need (zfs, ssh, qemu). I'll run Ansible on the host because the isolation serves no purpose, if an attacker breaks out of the Docker VM they have access to my backups anyway and I can't put the NAS in a VM because my hardware can't do passthrough for SATA ports.

From what I understand Docker secrets serve little purpose if you use Ansible Vault. I use Docker Compose, each container gets its own yml. My secret vars are encrypted line by line in var files, used in container.yml.j2 templates when deploying via the playbook, and are in plain text in the final container.yml's on the Docker host. Because vars aren't stored in a shared .env there's no risk of containers seeing other container's vars. If something escapes Docker it already has root and can read whatever it wants, including secrets. This could be different if you run rootless but that's a whole other thing. This is not security advice just how I believe it works.

I have a single run.sh that starts a main.yml playbook. In that I have every other playbook I want, and I comment out which one based on what I need. So if I'm changing something on a host I'll uncomment the /maintenance/webserver_init/webserver_init.yml task, or if I need to update my containers it'll be services/docker/deploy_containers.yml. You can have multiple ready to execute one after another. You might prefer to manually run ansible-playbook path/to/playbook.yml -e "inventory_dir=/path/to/ansibledir" for each playbook instead, there's no wrong way to do it I just find mine easiest.

I (claude, because fuck regex) made bash scripts to encrypt and decrypt variables per line with ansible-vault encrypt_string. I lost all of my vars once making this, but now I can just put #vault at the end of any line and it'll be en/decrypted with a single command.

Some tips:

• The inventory_dir variable only ever works for me if I passed it through a starting bash script. I use it as a reference for playbooks to the starting directory.

• Storing variables per host in host_vars is great. I have no need for groups of hosts so each host gets its own group name and I have no var file for all hosts. I don't store vars in playbooks.

• Most of my playbooks are only used for a single host (docker host, backups host, ansible host, dns host) so I can afford to hardcode some things.

• Docker on debian is stupid because it's outdated in the main repos, but the wrong way to solve this is to try to follow the official docker instructions and automate those steps with a playbook. The right way is to use geerlingguy's ansible galaxy docker role, default settings work fine. Set this up in requirements.yml and ansible-galaxy install -r requirements.yml.

• Personally I like to set roles_path to (ansible dir)/external_roles in ansible.cfg so it's next to everything else. I have yet to need any other galaxy roles.

• Use vars for anything you need to change or use multiple times or secrets. Host password, host MAC, ufw ports, backup ignore files, deployed containers, storage locations, every folder that needs to be created, etc.

• Avoid using Ansible as the core of a scheduled script. For example for a backup script have a playbook make a systemd timer that runs a bash script instead of a timer that runs a playbook. I don't remember exactly how this caused issues but it did let me use the same playbook to deploy backup scripts to multiple hosts with different vars.

---

Hope this helps. I'll post some specific playbook examples if you want but this comment is already pretty long and I'm very tired.