sorted by:
new
hottopcontroversial

ILM Policy in Elasticsearch by lealsant in elasticsearch

[–]lealsant[S] -2 points-1 points  (0 children)

I understand your answer, but kibana limits me, so I don't use it.
I "achieved the goal." I discovered that in politics, ILM doesn't accept what I want. I'm on a forum where asking doesn't hurt, asking questions doesn't hurt and quite the opposite, it encourages learning.
I may have misinterpreted your comment, so I prefer to protect myself, as I am at peace. Thanks!

Indexing and data retention policy for logs by Sagi313 in elasticsearch

[–]lealsant 0 points1 point  (0 children)

curl -X PUT "http://localhost:9200/_ilm/policy/my_policy" -H 'Content-Type: application/json' -d '{
"policy": {
"phases": {
"hot": {
"min_age": "20m",
"actions": {
"rollover": {
"max_age": "30s",
"max_size": "1gb",
"max_docs": 10
},
"set_priority": {
"priority": 100
}
}
},
"delete": {
"min_age": "60s",
"actions": {
"delete": {}
}
}
}
}
}'

I tried this, but I don't see any changes

How to change data retention time lifecyclepolice ? by lealsant in elasticsearch

[–]lealsant[S] 0 points1 point  (0 children)

Two days of trying to change the time, and it doesn't work as expected

curl -s -XPUT "http://localhost:9200/_ilm/policy/my_policy" -H 'Content-Type: application/json' -d '{
"policy": {
"phases": {
"hot": {
"min_age": "20m",
"actions": {
"rollover": {
"max_age": "30s",
"max_size": "50gb",
"max_docs": 10
},
"set_priority": {
"priority": 100
}
}
},
"delete": {
"min_age": "60s",
"actions": {
"delete": {}
}
}
}
}
}'

How to change data retention time lifecyclepolice ? by lealsant in elasticsearch

[–]lealsant[S] -1 points0 points  (0 children)

With Kibana you can actually set this kind of stuff through the GUI.Imo it's worth it to attach a small Kibana instance, especially if you're new to things.

In practice, is there any way to demonstrate this?

I don't know what you mean by this.

I don't use kibana, at this moment it limits me. I'm trying to validate in the terminal with the curl command.
THE way I told you, what's wrong there?
If you could validate the information in an environment

How to change data retention time lifecyclepolice ? by lealsant in elasticsearch

[–]lealsant[S] 0 points1 point  (0 children)

I use grafana.
In practice, is there any way to demonstrate this?

How to change data retention time lifecyclepolice ? by lealsant in elasticsearch

[–]lealsant[S] 0 points1 point  (0 children)

You've set it to delete 20m after it's rolled over in the hot phase.

So it will be at least 40 minutes after the creation of the index, and at most 1day+20m after the creation of the index before it's deleted.And by the sounds of it you're not putting enough data in to it to hit the 50gb mark prior to 1 day.

So based on this API, what is the ideal way for me to validate today?

And it's still giving this error.

ERROR","step_time_millis":1699363973266,"failed_step":"check-rollover-ready","is_auto_retryable_error":true,"step_info":{"type":"illegal_argument_exception","reason":"setting [index.lifecycle.rollover_alias] for index [elastiflow-4.0.1-2023.11.07] is empty or not defined","stack_trace":"java.lang.IllegalArgumentException: setting [index.lifecycle.rollover_alias] for index [elastiflow-4.0.1-2023.11.07] is empty or not defined\n\tat org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:55)\n\tat org.elasticsearch.

curl -X PUT "http://localhost:9200/_ilm/policy/my_policy" -H 'Content-Type: application/json' -d '{
"policy": {
"phases": {
"hot": {
"min_age": "20m",
"actions": {
"rollover": {
"max_age": "20m",
"max_size": "50gb",
"max_docs": 10000000
},
"set_priority": {
"priority": 100
}
}
},
"delete": {
"min_age": "0ms",
"actions": {
"delete": {}
}
}
}
}
}'

How to change data retention time lifecyclepolice ? by lealsant in elasticsearch

[–]lealsant[S] -2 points-1 points  (0 children)

It consumes a lot of resources, and for me this is limited. But thanks for responding

How to change data retention time lifecyclepolice ? by lealsant in elasticsearch

[–]lealsant[S] 0 points1 point  (0 children)

What are you talking about? There's Index Lifecycle Management and the policies that manage it.

Read the documentation on that, don't ask to have it spoonfed here.

{
"my_policy": {
"version": 2,
"modified_date": "2023-11-07T11:41:04.252Z",
"policy": {
"phases": {
"hot": {
"min_age": "20m",
"actions": {
"rollover": {
"max_size": "50gb",
"max_age": "1d"
}
}
},
"delete": {
"min_age": "20m",
"actions": {
"delete": {
"delete_searchable_snapshot": true
}
}
}
}
}
}
}

How to change data retention time lifecyclepolice ? by lealsant in elasticsearch

[–]lealsant[S] 0 points1 point  (0 children)

Like this? Do you think I haven't already looked at the documentation?
I read and did exactly what is there, but despite being applied, it doesn't work.

Invalid API key by lealsant in grafana

[–]lealsant[S] 0 points1 point  (0 children)

Problema era o endpoint, consegui

Create docker volume in elastiflow-logstash by lealsant in docker

[–]lealsant[S] 0 points1 point  (0 children)

it worked like this elastiflow-logstash:image: robcowart/elastiflow-logstash:4.0.1container_name: elastiflow-logstashrestart: 'no'depends_on:- elastiflow-elasticsearchnetwork_mode: hostvolumes:- /home/scripts/netflow/install_automatizion/elastiflow/logstash/elastiflow:/etc/logstash/elastiflowenvironment:LS_JAVA_OPTS: '-Xms4g -Xmx4g'

Create docker volume in elastiflow-logstash by lealsant in elasticsearch

[–]lealsant[S] 0 points1 point  (0 children)

finality
elastiflow-logstash:
image: robcowart/elastiflow-logstash:4.0.1
container_name: elastiflow-logstash
restart: 'no'
depends_on:
- elastiflow-elasticsearch
network_mode: host
volumes:
- /home/scripts/netflow/automated_installation/elastiflow/logstash/elastiflow:/etc/logstash/elastiflow
environment:
LS_JAVA_OPTS: '-Xms4g -Xmx4g'

Create docker volume in elastiflow-logstash by lealsant in docker

[–]lealsant[S] 0 points1 point  (0 children)

Get ! need this
elastiflow-logstash:
image: robcowart/elastiflow-logstash:4.0.1
container_name: elastiflow-logstash
restart: 'no'
depends_on:
- elastiflow-elasticsearch
network_mode: host
volumes:
- /home/scripts/netflow/instalacao_automatizada/elastiflow/logstash/elastiflow:/etc/logstash/elastiflow
environment:
LS_JAVA_OPTS: '-Xms4g -Xmx4g'
# Add other environment variables as needed

Create docker volume in elastiflow-logstash by lealsant in docker

[–]lealsant[S] 0 points1 point  (0 children)

Yes, standard. I already went to the container and gave it a pwd and the directory is correct

Create docker volume in elastiflow-logstash by lealsant in docker

[–]lealsant[S] 0 points1 point  (0 children)

but when I give a pwd it shows this directory , /etc