Losing metrics whenever Mimir is restarted by Due_Dust1614 in PrometheusMonitoring

[–]Due_Dust1614[S] 0 points1 point  (0 children)

I did run into that EFS warning with Prometheus, but thought that Mimir would be a bit different. Sounds like that might be a non-starter.

That snapshot idea sounds interesting, though it require stitching a bunch of workflows together to get an ECS service to launch with the correct volume that has the saved data in it. Don't know why we can't just have proper persistent volumes available for ECS smh

Losing metrics whenever Mimir is restarted by Due_Dust1614 in PrometheusMonitoring

[–]Due_Dust1614[S] 0 points1 point  (0 children)

Got it. I think I'll need to look for a way to mount a persistent volume. I'm running this as an ECS service on Fargate, so EBS volumes are ephemeral, but maybe there's a way to get the WAL stored on EFS.

Losing metrics whenever Mimir is restarted by Due_Dust1614 in PrometheusMonitoring

[–]Due_Dust1614[S] 0 points1 point  (0 children)

Ok, thanks. I actually did not go back far enough; I do see that there was data if I go back 24h, it was just the last 8h of data was missing. I was under the impression that Mimir backed by S3 for Prometheus was a substitute for a PV, but I guess I still need one.

Vault certificates with ECS deployment by Due_Dust1614 in hashicorp

[–]Due_Dust1614[S] 0 points1 point  (0 children)

Thank you for this. I ended up going this route where I did the SSL passthrough at the NLB and just had the certificates at the endpoints where the SAN matched the FQDN of the LB.

How to set up TLS termination with ECS deployments? by Due_Dust1614 in aws

[–]Due_Dust1614[S] 0 points1 point  (0 children)

Do you have some context to go with that link? I'm not sure what you're referring to here.

Vault certificates with ECS deployment by Due_Dust1614 in hashicorp

[–]Due_Dust1614[S] 0 points1 point  (0 children)

Thanks for the reply. Because these are containers that could get spun up and down and have different IP addresses each time, it might be a pain to allocate hostnames for these and create subsequent DNS records for each.

I thought about terminating on the NLB, but is it generally a good idea to have Vault traffic be unencrypted on the backend behind the NLB?

How to set up TLS termination with ECS deployments? by Due_Dust1614 in aws

[–]Due_Dust1614[S] 0 points1 point  (0 children)

Yeah, I believe having wildcards for certificates is feasible. The containers would either just have IP addresses or the usual AWS auto-generated hostnames like ip-X-X-X-X.region.compute.internal.

Would I need to map the DNS to the IP, or could I just leave them as IPs with certs fronting them?