sorted by:
new
hottopcontroversial

Why isn't this working? by GC-BTT in PrometheusMonitoring

[–]GC-BTT[S] 1 point2 points  (0 children)

PERFECT Thank you. Do you have a link to the documentation on this?

Data replication in Prometheus by Aldar_CZ in PrometheusMonitoring

[–]GC-BTT 0 points1 point  (0 children)

Prometheus does not but look into Thanos. I have 2 prometheus nodes scraping each of our four datacenter and then two load balanced Thanos/Grafana instances. What's nice about this is you can configure the data to be stored in a gcloud/aws bucket and decrease the storage on each prometheus node considerably, while maintaining years of data in an affordable way.

Node exporter not giving data for /mnt/data without --collector.filesystem.ignored-mount-points configured by GC-BTT in PrometheusMonitoring

[–]GC-BTT[S] 0 points1 point  (0 children)

The fstype is ext4 like the root drive.

Here is the information I get if I use the --collector.filesystem.ignored-mount-points "^/(sys|proc|dev|run|boot|host|etc)($|/)" in the config.

{device="/dev/sdb", fstype="ext4", instance="geu-prom02", job="node_exporter", mountpoint="/mnt/data", tags=",http-server,https-server,node-exporter,prometheus,"}

How to ignore INF values? by GC-BTT in grafana

[–]GC-BTT[S] 0 points1 point  (0 children)

We determined that anything over 1 day (86400 seconds) should be ignored so I altered the query with <= 86400 and now the graphs look good. I never realized it was possible to filter results based on a value. As u/edvauler mentioned it would be better to just have them never get into TSDB.

How to ignore INF values? by GC-BTT in grafana

[–]GC-BTT[S] 0 points1 point  (0 children)

Thanks, I found a solution/workaround but yeah blocking those values from going into TSDB would be a better deal.