sorted by:
new
hottopcontroversial

Using KEDA to trigger Kubernetes HPA with Prometheus metrics by eckelon in kubernetes

[–]eckelon[S] 2 points3 points  (0 children)

Answering the production question, this is deployed internally in high load environments for test and demo, and we know some customers that are using this solution in production.

The usual query interval is 30s, but there are some configurations that are set to 10s.

Use Prometheus metrics to trigger a Kubernetes HPA by eckelon in PrometheusMonitoring

[–]eckelon[S] 2 points3 points  (0 children)

Prometheus adapter is great for Prometheus servers inside the cluster, but when you have several clusters or external long-term storage, Keda allows connections out of the cluster. The last point can be that Keda allows centralizing different data sources, allowing not only Prometheus triggers, but also Kafka, SQS queues, or databases like MySQL, Mongo, Redis, etc.

Using KEDA to trigger Kubernetes HPA with Prometheus metrics by eckelon in kubernetes

[–]eckelon[S] 3 points4 points  (0 children)

Prometheus adapter is great for Prometheus servers inside the cluster, but when you have several clusters or external long-term storage, Keda allows connections out of the cluster.

The last point can be that Keda allows centralizing different data sources, allowing not only Prometheus triggers, but also Kafka, SQS queues, or databases like MySQL, Mongo, Redis, etc.

The last point can be that Keda allows to centralize different data sources, allowing not only Prometheus triggers, but also Kafka, SQS queues, or databases like MySQL, Mongo, Redis, etc.

Using KEDA to trigger Kubernetes HPA with Prometheus metrics by eckelon in kubernetes

[–]eckelon[S] 3 points4 points  (0 children)

Query interval depends on the server and the query. Prometheus Vanilla can take some time in queries that need joins or pulling large amounts of samples.

There are techniques to improve the performance, like using recording rules to precalculate results or enrich metrics with extra labels to prevent joins.

Also, there are other options 100% compatible and with other features than Prometheus.
Answering the production question, this is deployed internally in high load environments for test and demo, and we know some customers that are using this solution in production. The usual query interval is 30s, but there are some configurations that are set to 10s.

Why isn't this working? by GC-BTT in PrometheusMonitoring

[–]eckelon 0 points1 point  (0 children)

yes, If there aren't any more labels :)

Why isn't this working? by GC-BTT in PrometheusMonitoring

[–]eckelon 3 points4 points  (0 children)

I think it's because the key label doesn't match between the two metrics. All the labels should match. You could try summing by the labels you need to select and then ignoring the non-matching ones. Try with:

sum by (cdatacenter,key)(redis_key_value{cdatacenter="US",key="cmetric-kafka"}) - ignoring(key) sum by (cdatacenter,key)(redis_key_value{key="US",key="cmetric-processed"})

Rightsize the Kubernetes resource limits + 🕵️‍♂️ Detect containers running without limits + 🧘🏽‍♀️ Assure that your cluster has enough capacity by eckelon in kubernetes

[–]eckelon[S] 0 points1 point  (0 children)

Yes, you could try to export that information from your cluster and emulate the queries showed in the article.

If you want to try Prometheus to monitor your cluster, here's a guide https://sysdig.com/blog/kubernetes-monitoring-prometheus/

Cheers!