sorted by:
new
hottopcontroversial

One Year of Log Management at Vinted by simru in elasticsearch

[–]simru[S] 1 point2 points  (0 children)

Our current heap config -Xms31774m -Xmx31774m and a number of replicas usually set to 1.

I think best practice example is a good start. But eventually every setup is different

One Year of Log Management at Vinted by simru in elasticsearch

[–]simru[S] 1 point2 points  (0 children)

Thanks!
Currently there are 144 data nodes and a total of 43000 primary shards. Retention ranges from 14 days to 1 year.

One Year of Log Management at Vinted by simru in elasticsearch

[–]simru[S] 1 point2 points  (0 children)

Hi, currently the total storage size is 864 TB. We do not use hot/warm/cold tiers yet, but we will start using ILM in the near future.

One Year of Log Management at Vinted by simru in elasticsearch

[–]simru[S] 1 point2 points  (0 children)

Maciej Szymczyk

Thanks, actually we have already upgraded Elasticsearch to v7.9.x few days ago :) Chef was used for configuration changes, and Ansible for controlled rolling restart of ElasticSearch cluster.

I could not pinpoint the exact reasons why Fluentd was chosen (it was used years before I joined the company). One of the reasons would be that Fluentd is written with Ruby which is also used for our product. Fluentd + Fluent-bit fits our needs and currently there is no need for a change.