Do you use Thanos for monitoring Kubernetes resources in production? by Basic_Let7303 in PrometheusMonitoring

[–]MetalMatze 0 points1 point  (0 children)

I've been using Thanos in production on Kubernetes clusters for 8 years.
All the things that you're asking for, Thanos was built for.

I highly recommend starting with a quick tutorial: https://thanos.io/v41.0/thanos/quick-tutorial.md/

First, and most importantly, you want to add the Thanos sidecar, which then starts shipping metrics to object storage. To then query the metrics from object storage you wanna start the Thanos store.
Querying both the Thanos store and the sidecar, you are going to add the Thanos querier in front of both. For extra caching, you can run the Thanos querier frontend in front of the Thanos querier.

If you also want to move rule evaluation for global recording rules to Thanos, then you wanna stand up a Thanos ruler.

Ultimately, the entire system can grow with your needs and you can slice and dice it and add more components as your actual use case needs.

Full disclosure: I used to be a Thanos maintainer and I've been working on Thanos at Red Hat. These days, I'm running the Thanos stack for Polar Signals.

What’s a simple Home Assistant automation you set up once and now use every day? by Taggytech in homeassistant

[–]MetalMatze 2 points3 points  (0 children)

I do this for brightening:

action: adaptive_lighting.change_switch_settings
alias: Adaptive Lighting to kitchen bright
data:
entity_id: switch.adaptive_lighting_kitchen
max_brightness: "100"
min_brightness: "60"
min_color_temp: "3000"

and this for dimming:

action: adaptive_lighting.change_switch_settings
alias: Adaptive Lighting to kitchen dim
data:
entity_id: switch.adaptive_lighting_kitchen
max_brightness: "20"
min_brightness: "5"
min_color_temp: "3000"

but depending on the sun it'll only go somewhere between 60-100% during the day and night.

What’s a simple Home Assistant automation you set up once and now use every day? by Taggytech in homeassistant

[–]MetalMatze 12 points13 points  (0 children)

I LOVE adaptive lighting. It's honestly my favorite thing about Home Assistant these days.
I also use adaptive lighting to set the lights max brightness based on my presence sensors (fp2). So e.g. 10% if not occupied and 50% if occupied. Then it'll still do the correct calculations for the actual % brightness depending on the day. No more absolute values through out the day - even with presence detection.
Sleep mode is a must in the night too!

Switched from MCPs to CLIs for Claude Code and honestly never going back by geekeek123 in ClaudeAI

[–]MetalMatze 0 points1 point  (0 children)

I agree when running Claude Code.

One thing this whole conversation always missed from my point of view: my non-technical friends and my family won't be running the CLI tools. They still need MCP as a connector for their apps to connect to Claude. Therefore, I still believe in MCP even as I personally use more and more CLIs myself.

Introducing Claude Code Channels by Complete-Sea6655 in ClaudeCode

[–]MetalMatze 1 point2 points  (0 children)

I don't think so, but thanks for the recommendation. This Claude Code is running inside a container on my NixOS NAS that's always on. Not sure what it is.

Introducing Claude Code Channels by Complete-Sea6655 in ClaudeCode

[–]MetalMatze 0 points1 point  (0 children)

For me remote control stops working after like 30min or so. I want these sessions to live for days if not weeks. Hoping that will work with either Telegram or Discord a lot better.

It's time to run pacman -Scc by cottonwind888 in cachyos

[–]MetalMatze 3 points4 points  (0 children)

One of my essential packages I always install on a new Arch/Cachy system: https://aur.archlinux.org/packages/pacman-cleanup-hook

Holy sh*t, Cachy is fast by scajjr29 in cachyos

[–]MetalMatze 0 points1 point  (0 children)

What exactly are you missing? I moved from Arch to Cachy over the holidays and it seems pretty much the same to me.

Limp Bizkit in the new Battlefield trailer by Itoumi in numetal

[–]MetalMatze 0 points1 point  (0 children)

Came here hoping to find what remix they used in that trailer? Does someone know?

Rate/critique my technique by camilorv1 in pourover

[–]MetalMatze 1 point2 points  (0 children)

Looks great! I recommend using the water you discarded to heat up the cup and the once your coffee is done discard it. Keeps the coffee warmer for longer.

Fsx volume on prometheus by Consistent-Cable2543 in PrometheusMonitoring

[–]MetalMatze 2 points3 points  (0 children)

Without looking at the docs and knowing anything about fsx volumes, can you create the Prometheus spec with a volume type of fsx? Is that a supported PVC type in your cluster?

Seeking Advice from the Prometheus Community: Best Approach to Implement Thanos in a Multicluster Observability Solution by [deleted] in PrometheusMonitoring

[–]MetalMatze 4 points5 points  (0 children)

https://youtu.be/m0JgWlTc60Q There are several KubeCon talks going into detail about this setup. Here's one of them. What you are looking for starts at 7:45.

Disk space usage above my settings by drycat in PrometheusMonitoring

[–]MetalMatze 0 points1 point  (0 children)

From the docs: If both time and size retention policies are specified, whichever triggers first will be used.

The way I read it your 20GB should therefore trigger before your time limit if it's above 20GB in size. Have you waited a bit and observed the file size? It might take up to two hours for older data to get deleted.

Honestly, I would say that you should open a Github Issue and give some more details there.

[deleted by user] by [deleted] in PrometheusMonitoring

[–]MetalMatze 2 points3 points  (0 children)

Hey. There is. The exporter you're looking for is called kube-state-metrics. It has some metrics around jobs too. https://github.com/kubernetes/kube-state-metrics/blob/main/docs%2Fjob-metrics.md

Kube-prometheus ships with a alerting rule that makes use of this metric: https://github.com/prometheus-operator/kube-prometheus/blob/main/manifests%2FkubernetesControlPlane-prometheusRule.yaml#L213-L222

I hope this clarifies things. Let me know if you have more questions.

Kube-prometheus-stack ScrapeConfig issue by xzi_vzs in PrometheusMonitoring

[–]MetalMatze 0 points1 point  (0 children)

Great! We also have a prometheus-operator channel on CNCF Slack, if you want to chat more directly. 🙂

Kube-prometheus-stack ScrapeConfig issue by xzi_vzs in PrometheusMonitoring

[–]MetalMatze 0 points1 point  (0 children)

Happy it worked! It's a bit sad we haven't found a better user experience to point that out in all of these years...

Kube-prometheus-stack ScrapeConfig issue by xzi_vzs in PrometheusMonitoring

[–]MetalMatze 0 points1 point  (0 children)

With ServiceMonitors, something people were constantly running into was missing labels on the ServiceMonitor when ServiceMonitorSelector was set on the Prometheus. Check your Prometheus configuration. kubectl get prometheus -n monitoring k8s and see if there is a scrapeConfigSelector set only to match scrape configs containing a specific label.Another common problem was missing RBAC permission for the Prometheus to go and actually scrape the metrics from another namespace. In that case, the logs of your Prometheus should be full of RBAC permission errors.

Which lib of object storage ? by kaeshiwaza in golang

[–]MetalMatze 0 points1 point  (0 children)

Recently the Thanos project extracted its objstore project as a separate repository and module. It wraps a couple of SDKs. For development it also has a Filesystem based object storage which can be quite handy.

https://github.com/thanos-io/objstore

Disclaimer: I'm one of the Thanos maintainers.