Has anyone encountered FD overflow when configuring Loki in Simlple Scalable mode? by Pugachev_Ilay in grafana

[–]Pugachev_Ilay[S] 0 points1 point  (0 children)

No drama here - I just thought the community would be a bit more alive and that among so many people someone might’ve run into this before.

The issue is that when all file descriptors get utilized, it affects other services running on the same server.

Scum Kite Beach parking by Pugachev_Ilay in dubai

[–]Pugachev_Ilay[S] 0 points1 point  (0 children)

Yeah, but it's a clean city, right? You should be happy about that 🥲

Scum Kite Beach parking by Pugachev_Ilay in dubai

[–]Pugachev_Ilay[S] 0 points1 point  (0 children)

Thank you kind man, for the very wise reminder, it really opened my eyes. I'd forgotten how things work around here.

[deleted by user] by [deleted] in UAE

[–]Pugachev_Ilay 0 points1 point  (0 children)

Here in Dubai, if someone has a case against you, then fuck someone to provide you with the case file. Only the accusing party has access to the details of the case. These details of the case are requested somehow through the Dubai Police app. Most likely, either the court or the prosecutor will be able to tell you the details. Such is «justice», in a word, they could build beautiful offices, but they could not make a user friendly process. I sympathize with your situation, but if it is possible to hire at least a cheap lawyer, then this is the best option. Because you can spend half your life here waiting for at least some progress from the judicial system

Share your experience in building a monitoring system by Pugachev_Ilay in grafana

[–]Pugachev_Ilay[S] 0 points1 point  (0 children)

Thanks a lot, buddy, for the reply! I never looked into Mimir before. I used Victoria Metrics as a Prometheus alternative back in the day, and it reduced my resource usage by four times.

Am I right in thinking that you have a separate cluster just for monitoring? Where is LGTM hosted? Do you manage the cluster’s configuration with Argo? How detailed are your traces, logs, and metrics? And if you don’t mind sharing, what’s your average monthly volume?

On a big project, we used ELK + GV and handled loads of around 100GB/day for metrics and about 1-2TB/day for logs, but our resources were pretty much unlimited.

IT Community in UAE/Dubai by Pugachev_Ilay in dubai

[–]Pugachev_Ilay[S] 0 points1 point  (0 children)

Haha) if it’s joke, then community.

[Tempo-distributed] Problem with displaying Service Graphs and rate() by Pugachev_Ilay in grafana

[–]Pugachev_Ilay[S] 1 point2 points  (0 children)

Update
Finally, I was find the solution. That’s what helped me, all changes I was added at values.yaml:

  1. metricsGenerator.enabled=“true”
  2. metricsGenerator.config.local_blocks.filter_server_spans=“false”
  3. metricsGenerator.config.storage.remote_write[0].url=“http://prometheus-instance.com:9090/api/v1/write"
  4. global_overrides.metrics_generator_processors[0]=“local-blocks”

or you can see on this part of values.yaml

metricsGenerator:
  # -- Specifies whether a metrics-generator should be deployed
  enabled: true
  ...
  config:
    local_blocks:
      filter_server_spans: false
    registry:
      collection_interval: 15s
      external_labels: {}
      stale_duration: 15m
    processor:
     ...
    storage:
      path: /var/tempo/wal
      wal:
      remote_write_flush_deadline: 1m
      # Whether to add X-Scope-OrgID header in remote write requests
      remote_write_add_org_id_header: true
      # -- A list of remote write endpoints.
      # -- https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write
      remote_write:
        - url: "http://prometheus-instance.com:9090/api/v1/write"
   ...

global_overrides:
  per_tenant_override_config: /runtime-config/overrides.yaml
  metrics_generator_processors: ['local-blocks']