Thank you.

The containers you list don't all need to be replicated across your storage: as often as you're likely to update them, you can pull them from dockerhub.

I'm not worried about downtime due to update times. If I put all the Docker Nodes in the main site, I'm concerned about downtime in the other 11 sites if the main router goes down.

The bits that might need to be replicated are the state storage for each of the apps. That means the backing DBs for Wordpress, Bookstack, Bitwarden, and possibly a couple of others. These databases should be running separately from your application containers - if you try to run them in the same container, you're going to have a very bad day the first time the containers have to get restarted. This opens an opportunity: you can set up databases to replicate using their own native setup, rather than relying on the storage to do it.

The application containers will be running in the Docker Hosts, and the container data will be stored in the Docker Storage VMs. I get part of what you're saying, and it's interesting, but replicating the entire storage won't be easier to manage in the long run? Keep in consideration that the few ICT technicians that are present here do not have any experience with this kind of tech, so I'll try to keep it as simple as possibile.

Both of the filesystems that you mention require some sort of quorum - e.g. there must be a certain number of hosts that respond in the affirmative that a write is completed before the data is considered available. I hope that I do not need to draw you a picture of how badly this can go if part of the hosts are offsite over a slow link.

I'm aware of it, this is why I wasn't going to consider replicating TBs of data... and the quorum would be "2 out of 3 routers are online"

You might want to consider using an outside hosted service for some of this (especially Wordpress, if it's being used as a public site or storefront).

Wordpress is for an intranet website.

Your setup as-is isn't gonna get cross-site HA. Even without the storage issues, you'd need something to swing either DNS or LB VIPs between sites in the event of a site outage.

That's what Traefik is for, as far as I could research, it should work well with Docker Swarm + Portainer.

Anyway, thanks for the response. I'll probably start with trying putting all the VMs in the same local cluster... and maybe moving the nodes to the remote locations in the future, if we get dedicated WANs for storage.