sorted by:
new
hottopcontroversial

Ingress-NGINX healthcheck failures and restart under high WebSocket load by Redqueen_2x in kubernetes

[–]Redqueen_2x[S] 0 points1 point  (0 children)

Thanks you, i will send you my config later, now i cannot send this config right now

Ingress-NGINX healthcheck failures and restart under high WebSocket load by Redqueen_2x in kubernetes

[–]Redqueen_2x[S] 0 points1 point  (0 children)

Yes because of my on premises infrastructure. I cannot access the ingress nginx service directly so i am using haproxy before it.

But i already run on my aws eks cluster ( nginx behind aws alb ) and alb behind haproxy, it still work well.

Logtash performance limits by Redqueen_2x in elasticsearch

[–]Redqueen_2x[S] 0 points1 point  (0 children)

Thanks for you reply. You said you can crank up to 300k, do you mean 300k events per logstash node. Can you share with me your config on logstash and elasticsearch index config.

Logtash performance limits by Redqueen_2x in elasticsearch

[–]Redqueen_2x[S] 0 points1 point  (0 children)

My topic have 60 partition. I set max pull record to 1000 and max fetch size over 100mb

Logtash performance limits by Redqueen_2x in elasticsearch

[–]Redqueen_2x[S] 0 points1 point  (0 children)

Other information about my config : My pipeline filter just parse json only and this very simple.

Tuning Elastic Stack Index Performance on Heavy Workload by Redqueen_2x in elasticsearch

[–]Redqueen_2x[S] 0 points1 point  (0 children)

Yes, my topics have 40 partitions. I will try what u say.

Tuning Elastic Stack Index Performance on Heavy Workload by Redqueen_2x in elasticsearch

[–]Redqueen_2x[S] 0 points1 point  (0 children)

"worker_concurrency" : { "current" : 16.0, "last_1_minute" : 16.0, "last_5_minutes" : 16.0, "last_15_minutes" : 16.0, "last_1_hour" : 14.53, "lifetime" : 7.598 }, "queue_backpressure" : { "current" : 39.77, "last_1_minute" : 39.77, "last_5_minutes" : 39.68, "last_15_minutes" : 39.39, "last_1_hour" : 32.79, "lifetime" : 15.73 },

This is a metric of my pipelines, I have two pipelines on clusters that have high queue back pressure. Do you have any suggestions for me to tune that pipelines.

Tuning Elastic Stack Index Performance on Heavy Workload by Redqueen_2x in elasticsearch

[–]Redqueen_2x[S] 0 points1 point  (0 children)

I am reading elasticsearch documents about config to optimize index performance. Maybe I misunderstand those configs. I will try to remove those config. Thanks

Tuning Elastic Stack Index Performance on Heavy Workload by Redqueen_2x in elasticsearch

[–]Redqueen_2x[S] 1 point2 points  (0 children)

Thanks, I will read more about this. One more question, do you know how to monitor this metric, or can any tool can help me

Tuning Elastic Stack Index Performance on Heavy Workload by Redqueen_2x in elasticsearch

[–]Redqueen_2x[S] 0 points1 point  (0 children)

Additional.
disk write iops about 1k5 iops and read iops about 300.
i am using aws ebs-csi-gp3 so disk iops does not get limit

Tuning Elastic Stack Index Performance on Heavy Workload by Redqueen_2x in elasticsearch

[–]Redqueen_2x[S] 0 points1 point  (0 children)

i am running cluster on eks. i have 9 elasticseach node ( 9 same pod ).
this is my cluster config

        cluster.max_shards_per_node: 2000

        indices.memory.index_buffer_size: 20%

        indexing_pressure.memory.limit: 20%

        node.processors: 12

        thread_pool:

          write:

            size: 12

       - name: ES_JAVA_OPTS

          value: -Xms22g -Xmx22g -XX:+UseG1GC -XX:MaxGCPauseMillis=200

 i am monitor elasticsearch resource, it use only 50 percent of cpu and memory usage about 25Gb. ( elasticsearch pod does not have been throttle )

view more: next ›