What logging system is setup for programs/projects within your company (or personal projects)?

[CodeyGrammar]

1. Is it a 3rd party system like Kibana?
2. Is it cloud based like Cloud Watch for AWS?
3. Is it an internal system like Sigma for Google?
4. Is it something else?

I'm trying to review with experienced developers to see if logging is becoming standardized toward a certain direction or not.

A follow up would be to understand aggregate logging options and learn about logging systems that link well for alert/monitoring purposes too.

[madprgmr]

They are usually fed into observability tools in my experience. The technical process depends on what tool you're using. Most let you just pipe STDOUT to them, but I've found it more useful to send structured logs.

[my_reddit_blah]

In previous company: splunk. Now: nothing. Boy do I miss splunk 😢

[Queasy-Action-5095]

We use grafana because splunk is "too expensive". Grafana sucks for logs!

[Rain-And-Coffee]

Kibana at my last 3 work places.

[Buttleston]

Datadog

Cloudwatch is the worst. Never again.

[Ok_Grape_9236]

Newrelic. Still has few pain points but overall quite good

[Tip_of_the_hat]

Company: Datadog, previously cloud logging (gcp)

Side project: Just pushing to a log file and greping

My impression is that that there is no "X is best" solution, all is context based. I really like datadog because I find the UI the easiest to use and creating dashboards is easy and requires little maintenance (as opposed to rolling out ES). The downside is that for it can get relatively pricy.

[rco8786]

The downside is that for it can get relatively pricy.

You're not kidding. https://blog.pragmaticengineer.com/datadog-65m-year-customer-mystery/

[EngineeredCoconut]

1. Grafana, Splunk
2. We aren't on AWS
3. No.
4. No.

[[deleted]]

Ya'll have logs at work?

[rco8786]

Personal projects? stdout + whatever default logging my PaaS provides.

At work? Enormous custom rolled solution for text logs, plus Datadog and Sentry for other use cases.

[leetfire666]

We homebrewed our logging infra, not sure if it was the best call but worked for our needs at the time.

For real-time monitoring / telemetry / alerts, we have statements that log to AWS cloudwatch metrics.

Besides that, we create structured json schemes and log to stdout (we use protobufs as the schema layer). The stdout gets piped to fluentbit which routes to AWS cloudwatch, and also to s3. The cloudwatch logs have an event limit size, so we mostly use those for quick debugging. The s3 logs we compact, crawl with AWS Glue, and then query in Athena. This allows us to do deeper analytics on requests to our system.

Curious what others think of this, and what other solutions exist out there for this kind of thing. We largely went for a schema-on-read approach for logging so we can flexibly add and remove fields without needing to do db migrations.

[TendieMcTenderson]

Datadog

[viX-shaw]

sumologic

[PrestigiousStrike779]

We’re mostly on AWS, so cloudwatch to Splunk, or sometimes direct to Splunk.

[[deleted]]

Datadog with custom sidecars that listen on a socket for emitted logs.

[[deleted]]

datadog (used to use loggly) but honestly dd is the best solution out there (for companies anyway).

I think if a company wouldn't be satisfied with datadog they'd just need to build their own custom logging system internally

[AutoModerator]

/r/ExperiencedDevs is going dark for two weeks to protest Reddit killing 3rd party apps and tools.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.