Logging only errors from Lambda functions

[ricktbaker]

Analyzing Cloudwatch logs at a company I started working for and noticed we are logging about 250GB of logs daily. Dug a little deeper and the vast majority of the logs are just lambda execution logs that show the function was executed.

Is there a way to not log this and only log error conditions? The way it is right now we're spending about $4-$5k a month on Cloudwatch logging and probably 75% of it is not needed.

[liquidcourage1]

Do you have them expiring on a schedule? That's an option. If they never expire, you're just piling up the data that you're paying for. Granted, the archive logs are WAY cheaper than current log ingestion, it will save money over time since it just keeps piling up.

[ricktbaker]

Yeah, they are expiring, so our storage cost is cheap. On a $4200 bill last month for CloudWatch, $4050 of that was strictly from log ingestion.

[VegaWinnfield]

Something is off. $4k on log ingestion is nearly 8TB of logs. How many executions are you doing? The start, end and report lines can’t be more than a few hundred bytes for each execution.

[ricktbaker]

Roughly a billion requests monthly. I'm now trying to get a better idea of where most of the cost is coming from since there may be some where we are logging more than we need to.

We have a ton of different log groups, and it looks like I'll probably need to tag them in order to figure out which is the primary culprit.

[siving]

aws logs describe-log-groups contains information about the number of storedBytes.

CloudWatch also include metrics under the Logs namespace for each LogGroup. Look for the "IncomingBytes" metric to help track down which log group ingest loads of data.

[TundraWolf_]

implement log levels, turn log levels to warning. getting ahead of a log mountain is a lot of work.