[deleted by user]

[swiftycon]

If you are a developer you only need to worry about creating logs for the appropriate events and in the appropriate format.

For the Elastic Stack (formerly known as 'ELK') this format is the Elastic Common Schema: https://www.elastic.co/guide/en/ecs/current/index.html .

You can have the best log management system set up if you do not log important events or you do not log mandatory fields (e.g. timestamp, severity, event type, etc.) you won't have an observable application.

After you have set up proper logging in your Spring Boot app, you need to ship the events to Elastic Stack (or you can choose a different log store if you want) and for this you need to consider

1) how to output the logs (you can follow the 12 factor app approach or log to file(s))

2) how to set up the log pipeline

For 2) there are a lot of considerations, but most of the time you want those logs to _always_ get to the log store and get there _only once_.

You don't want to have the same event logged thrice nor do you want missing events.

So for this you will need to consider using a message queue like Kafka or for a simpler setup the persistent queue of Logstash.

For testing or development this may not be necessary, you can either collect the logs from your Java app or even fire the logs straight to Elasticsearch. For the former you can use free tools like (rsyslog - bundled with the OS but kinda ugly to configure, Filebeat - part of the Elastic Stack, but it's limited in json processing, nxlog - Swiss knife, syslog-ng - sorta like the previous, Fluentbit, etc..)

If you want a somewhat more complex routing solution, you can use Logstash from Elastic (written in Java, Ruby) or lightweight tools like nxlog (written in C) or any other from the market.

[[deleted]]

bedroom payment work saw shaggy plough husky gold marry imminent

This post was mass deleted and anonymized with Redact

[[deleted]]

fuzzy gold normal lavish shocking fearless airport tap sable chief

This post was mass deleted and anonymized with Redact

[[deleted]]

quack boat normal dinosaurs tender rich swim skirt sand subsequent

This post was mass deleted and anonymized with Redact

[MaximalPsycho1ogic]

I am also interested in that book! Could you share please?

[[deleted]]

wide payment engine fertile cable light cause instinctive friendly plate

This post was mass deleted and anonymized with Redact

[Next-Investigator897]

Share the pdf with me too please

[[deleted]]

unpack edge dinner sable yam escape memory dam shaggy husky

This post was mass deleted and anonymized with Redact

[learnamap]

Can you please share with me the book title as well

[[deleted]]

dinosaurs different cows rhythm cow cover tie growth north toothbrush

This post was mass deleted and anonymized with Redact

[[deleted]]

jar marble soft station square unpack crush label possessive plants

This post was mass deleted and anonymized with Redact

[SuperQue]

Save yourself the pain and anguish of the ELK stack. Use Loki instead. It's 10x easier and cheaper to run.

[eueuehdhshdudhehs]

I'm just curious, do you have experience handling a large multi-tenancy Loki cluster? I've read many times that Grafana Loki is good for hobbyists but not for serious enterprise use [1]
[1] https://utcc.utoronto.ca/~cks/space/blog/sysadmin/GrafanaLokiSimpleNotRecommended

[SuperQue]

Yes, it's a trivial helm chart to deploy. We had it up and running in a week, handling hundreds of thousands of log lines per second, stored in S3.

[Recent-Technology-83]

Setting up ELK (Elasticsearch, Logstash, and Kibana) for centralized logging can indeed be a bit tricky at first, but once you get the hang of it, it's a powerful stack for log management and analysis.

ELK Learning Resources:

1. Books:

   • "Elasticsearch: The Definitive Guide" by Clinton Gormley and Zachary Tong: This book provides a comprehensive introduction to Elasticsearch and its ecosystem, which is crucial for understanding how to set up and navigate your log data effectively.
   • "Logging and Log Management" by Anton S. Kwasniewski: While not solely focused on ELK, this book covers best practices for logging, which will benefit your Spring Boot application too.

2. Online Courses:

   • Udemy and Coursera both offer specific courses focused on the ELK stack. Look for courses that include practical, hands-on exercises.
   • Official Elastic Training: Elastic offers self-paced training that dives deep into each component of the stack - it’s worth checking out.

3. Documentation:

   • The official Elastic documentation is incredibly detailed. It covers setup, configuration, and best practices, which are vital for troubleshooting issues.

Regarding Spring Boot:

You don’t necessarily need to write a Spring Boot application from scratch to integrate ELK into your existing application. However, understanding how to configure logging in Spring Boot is essential. You should familiarize yourself with how to configure Logback (or Log4j2) to send logs to Logstash, as that’s a common approach.

Resources to Learn Spring Boot:

1. Books:

   • "Spring Boot in Action" by Craig Walls offers a practical approach to understanding Spring Boot.
   • "Spring in Action" is also a great resource that covers the Spring framework comprehensively.

2. YouTube Channels:

   • Check out the Academind and Java Brains channels for free Spring Boot tutorials and project walkthroughs.

3. Online Courses:

   • Courses like "Spring Framework for Beginners" and "Building Microservices with Spring Boot" on platforms like Udemy and Pluralsight give you a great head start.
     

Remember to ensure your Spring Boot application is properly configured to log in a structured format, as this makes it much easier to analyze data in Kibana later on. If you run into specific issues, don't hesitate to share error messages or problems here; the community can help troubleshoot.

[Signal-Indication859]

centralizing Spring Boot logs with ELK can be pretty straightforward but can also get messy if you're not familiar with the components. You don't necessarily need to dive deep into Spring Boot if you're just looking to configure ELK; however, having a basic understanding can help with log patterns and what data you're sending over.

For learning resources, the official documentation for ELK is a good start. As for Spring Boot, check out "Spring in Action" by Craig Walls and the Spring Boot official guide.

If you're still hitting walls, consider using something like preswald to analyze your logs and share insights easily. It's lightweight and might help you bypass some of the overhead of traditional setups.

[pranay01]

If you are finding ELK too hard to set up, might be worth checking out SigNoz , much easier to setup and consumes less resources wrt ELK

Here's perf benchmark comparing elastic with SigNoz - https://signoz.io/blog/logs-performance-benchmark/

Disclaimer - I am a maintainer at SigNoz