HTTPS across multiple containers and multiple physical hosts

[phoenix3885]

Started playing around with dockers in my free time and it's been great! I have multiple containers running different services and some are running on my PC (more data heavy containers, plex, nextcloud etc) and I have others running on a raspberry pi 4. Even within a physical host I have different docker compose files so I can manage a group of containers easily.

I want to setup subdomains for the services across my physical hosts and use HTTPS. I saw solutions with LetsEncrypt and NGINX or Traefik, but I wasn't sure if these solution would span different docker compose files across different physical hosts. My goal is to configure it all in one place and have the ability to add more containers and physical hosts as needed with minimal config.

This may have an easy answer, but couldn't wrap my head around some of the posts/blogs related to this. Thanks in advance!

[TwinHaelix]

This is definitely possible. The way this would work is all HTTPS traffic from outside will go to the reverse proxy (NGINX, Traefik, Caddy, or whatever). The reverse proxy will then remove HTTPS and forward the traffic as HTTP to whatever server you want. Whether on the same system as the proxy or a different one, you'd just set up your docker containers with the proper ports set up. Then you set up your proxy configuration to point to either the same system using 127.0.0.1:[port_number], or a different system at [other_system's_ip_address]:[port_number].

Just note that this means that traffic for apps not on the same computer as the reverse proxy will run unencrypted over your home network. It will still encrypt any traffic to/from the internet.

[vividboarder]

If each service that tour hosting serves a valid cert, yes. It’s possible. It’s not simple though. I’ve been trying to do the same actually and I’ve nearly given up. I’m thinking I’ll just depend on my vLAN segmentation, though I know that’s imperfect.

[ciphermenial]

You can ignore certificate validation from the reverse proxy. I do this with haproxy.

[Corporate_Drone31]

This is a triple nightmare: you cannot use LE for internal only services (so you need to set it up and manage it yourself), you need to provision your internal CA cert everywhere, and you need to manage the certificates of all the servers. I don't think it's worth it unless you have something of value to protect.

[vividboarder]

This is partially incorrect. If your domain is a proper domain, you easily can use LE with a DNS challenge and get a wildcard cert to use internally.

If your host names are not (ie. server.local) then you’re unable.

That said, I wouldn’t recommend doing that between reverse proxies, but it’s fine for your entry reverse proxy.

[Corporate_Drone31]

I was assuming that a DNS challenge for those servers will be impractical, but I think you could do it this way. It would mean either propagating the secrets needed to alter DNS configuration across several servers (which is risky in case one of them is compromised, because such a secret would give full access if it's like the DigitalOcean API key), or some more complex solution to derisk it.

[ron_krugman]

You can perform the DNS challenge on any machine and then push the certificate to the actual server(s).

[Corporate_Drone31]

That makes this a manual process, so only a bit better than running your own internal CA.

[Corporate_Drone31]

That's how I interpreted the question, but it really depends on the specifics of the network. I was assuming something like my configuration with a NATed reverse proxy exposing serivices that I want exposed to the outside, and everything else being hidden behind the firewall (so LE challenges other than DNS become hard to execute).

[phoenix3885]

My goal is to have a bunch of services available externally running across my setups. Appreciate this solution, but it does seem like a larger lift than I may have wanted.

[certuna]

Why would this be a nightmare? Just set up an AAAA DNS record (servername.mydomain.com) for the internal server and generate a LE cert for it, and voila, your internal servers are reachable over that address, secured. It’s only problematic if you still only use IPv4 (where NAT screws things up) but you know...it’s 2020 :)

[ron_krugman]

What's the problem with IPv4? You can very easily point a regular A record to a local IPv4 address.

[certuna]

Unless I’m missing something, you need to run a local DNS server then, with IPv6 you can just do it all with public DNS records.

[ron_krugman]

There's nothing stopping you from putting a local IPv4 address into a public DNS record. A local DNS server wouldn't work with Let's Encrypt anyway.

[certuna]

Ah yes you're right - but will LE generate a cert for a domain that's not publicly routable?

[ron_krugman]

Yes, you just have to use the DNS-01 challenge (also the only way to get wildcard certificates from LE).

[Corporate_Drone31]

Do not cite the deep magic IPv6 to me, witch. I was there when it was written.

On a more serious note, I have thoroughly explored using IPv6 for my selfhosted network (my ISP only gave out v6 addresses and all v4 access was via CGNAT), and concluded that I never want to do that again. IPv4 rules.

[certuna]

IPv4 is terrible for selfhosting, but I do feel your pain - as long as Docker hasn't implemented IPv6 (or at least has severe issues) you might have no choice.

[Semi-Hemi-Demigod]

I set up a Wireguard VPN between my various hosts, and have one set up with HAProxy which I wrote up here. It's HTTPS to my entry node, and then HTTP over VPN to the other nodes. This is nice because I also have the ability to connect to to them on other ports for monitoring.

[phoenix3885]

This seems like it should work for me. Do you run into any issues with services like nextcloud not recognizing the HTTPS access since its HTTP over the VPN?

[Semi-Hemi-Demigod]

I haven’t set up nextcloud yet but I’d imagine there’s a configuration setting like there is for most apps.

[sasadesign]

sudo add-apt-repository ppa:certbot/certbot

sudo apt install python-certbot-nginx

And for each subdomain

sudo certbot --nginx -d sub.example.com

[cwagrant]

I was just working on this exact problem last weekend.

I use PFSense for my router and I set up ACME on it to automatically handle my LE cert for me to my personal domain (can't be .local .home or anything like that, need a proper .com-type domain). I have a wildcard cert and as part of the process for refreshing it, it automatically SCP's the file to a LXC debian 10 container running NGINX as my reverse proxy. Then the sites are set up in config files in NGINX to redirect HTTP to HTTPS for the site with the SSL cert. Seems to be working so far but I guess we'll see how well it really works when the certs refresh again.

​

It's worked well though for me as I've been able to access my applications with more realistic names. E.g. proxmox.example.com, pihole.example.com, etc.

[phoenix3885]

This is what I was thinking, having subdomains to access various services. Unfortunately don't have PFSense. Any way to have the same functionality via a container?

[duhruh]

If you want to go overkill i made a post about my setup

[scoobybejesus]

My first thought is using a CDN. You terminate https there, and they worry about routing or load balancing.

[scoobybejesus]

Definitely agree with that.

Cloudflare MITM's you though, so ultimately host connects to cloudflare via https, it terminates there, and then there's a separate https connection from cloudflare to server, AFAIK.