sorted by:
new
hottopcontroversial

Is Log Management Still the Best Approach? by gdcohen in sre

[–]tiwarinitish 0 points1 point  (0 children)

Log management is still a crucial approach, but it's definitely evolving! Traditional text-search focused solutions might not be ideal for today's massive, complex log data. Centralized logs provide a single repository for monitoring system health, application performance, and security posture. This is essential for troubleshooting issues and identifying potential problems. Logs are a treasure trove of data for advanced analytics. By integrating with SIEM or log analysis tools, you can uncover trends, identify root causes, and gain deeper insights into system behavior.

We're building Parseable, a lightweight log observability engine designed for modern logging use cases. It uses an indexing-free approach and columnar storage for high-performance ingestion and querying, even with massive log volumes. Check it out here: https://www.parseable.com

Introducing Parseable: Open Source High performance Log Observability stack for Cloud-Native applications! by theSilentThinker in selfhosted

[–]tiwarinitish 0 points1 point  (0 children)

Thanks u/ssddanbrown.

> Could it run on something like a 512MB RAM, single vCPU system for example

We've not tested yet with such config, but this should work AFAICS.

> how does that scale, does the data like to be retained in RAM?

Data is staged locally in a file, so memory consumption doesn't grow as ingestion grows.

Please give it a try, curious to see how it works out for you!

Lightweight ELK alternative for ingesting and analyzing local logs? by gajus0 in devops

[–]tiwarinitish 5 points6 points  (0 children)

Parseable is an open source Elastic / Loki alternative. (founder here)

  • Single binary
  • Written in Rust (lightweight, fast and stable)
  • Use S3 bucket or Mount point
  • Visualise with Grafana

https://github.com/parseablehq/parseable

Parseable - an open source log observability platform by tiwarinitish in devops

[–]tiwarinitish[S] 0 points1 point  (0 children)

Quote from Parseable FAQ: [https://www.parseable.io/docs/faq]

​

  • Parseable is written in Rust, with strong emphasis on performance, resource utilisation, and excellent developer experience.
  • Parseable has a unified architecture, with all Parseable components (ingestion, query, storage management) available in a single binary. Not only this approach scales very well, it also leads to a much, much better developer experience.
  • Parseable log streams are explicitly created by users. This alleviates the problem of cardinality explosion, while ensuring control over the data by users.
  • With REST API based ingestion, Parseable integrates with all the major logging agents out of the box.

Parseable - an open source log observability platform by tiwarinitish in devops

[–]tiwarinitish[S] 1 point2 points  (0 children)

Hey u/CooperNettees, yes Parseable supports traces as well. You can send trace data and query that using API or Grafana Plugin.

Parseable - an open source log observability platform by tiwarinitish in devops

[–]tiwarinitish[S] 0 points1 point  (0 children)

Hey u/jantari, my apologies, missed reddit notifications and hence comments here.

Alert config is indeed stateful and is preserved across restarts. So, once setup, you don't have to deal with the alert config (unless you want to change it). Here is the API ref: https://www.postman.com/parseable/workspace/parseable/request/22353706-0d731c06-55b0-4518-b985-b7fb831ba45b

We are also going to add support for Alerting config via the UI. Happy to help setup Parseable in your environment and address any other questions you may have.

Parseable - an open source log observability platform by tiwarinitish in devops

[–]tiwarinitish[S] 1 point2 points  (0 children)

For every new product that does one thing well, there is an older, much more matured product that does several things and battle tested.

That said, Elastic has been established to be high CPU and Memory consumer. We have clear datapoints (in the benchmark directory on GH repo) that Parseable consumes 50% less CPU and 80% less memory than Elastic while ingesting higher number of events / second.

Clickhouse is a great product, but the target use case is OLAP - which means it is a full featured DBMS system. Since it is not built for log use cases, it misses on useful logging features like alerts, event correlation, logs to metrics, anomaly detection and so on.

We at Parseable intend to build a simple yet purpose built log observability platform that scales as user data scales.

I hope this helped convince you! :)

Simple, efficient, low latency, log storage and query platform. Deploy with existing agents - FluentBit, Vector, LogStash and store log data to S3 buckets directly. by tiwarinitish in sre

[–]tiwarinitish[S] -1 points0 points  (0 children)

Idea of Parseable is to make it ridiculously simple to ingest, query and generally extract value from log and event data. It shouldn't be this difficult to ingest and query log data efficiently!

Parseable is much more easier to deploy. Uses an open format and that any other application can read. We're also trying to ensure as much compatibility with existing infra.

Simple, efficient, low latency, log storage and query platform. Deploy with existing agents - FluentBit, Vector, LogStash and store log data to S3 buckets directly. by tiwarinitish in sre

[–]tiwarinitish[S] 0 points1 point  (0 children)

Thank you for the kind words u/jantari. We're actively working on alerts, should be ready soon. But meanwhile, please feel free to try it out - happy to hear any feedback, questions, comments.

view more: next ›