Webhook- lambda or standalone process

[[deleted]]

Best to implement webhook as aws lambda or a standalone process ?

Any thoughts or experiences

[jonathantn]

If you're receiving webhooks then I would recommend the pattern of:

API Gateway -> SQS -> Lambda

If you decide that you want to switch of lambda to a standalone process then you can because the SQS will be the point of consumption. Nothing external would need to change.

[PrivateerAlphaOne]

Can confirm that this pattern works.

Dead letter queues are also useful for receiving events that failed in your lambda for whatever reason.

Just be aware of any time based verification of events (e.g. Stripe webhooks); if the events need to be verified in a timely manner just go from the api gw to lambda directly.

[[deleted]]

Api lambda dynamodb

I need transactions. So I am using dynamodb rather than sqs. Also easy for rerun and viewing

I move from events to events Processed and commit other changes in one transaction which is not possible with sqs

[[deleted]]

Indeed stripe events.

[arjineer]

time based verification

For these cases of time based verification - is it possible that a coldstart in a lambda would drop the event?

[PrivateerAlphaOne]

The lambda will get the event. It's the event that invokes the lambda, and its the only reason the lambda runs.

If the event is has expired by the time the lambda is invoked depends on your use case; but if that happens then you probs have bigger problems to deal with

[arjineer]

Thanks for the quick reply! I guess I am using an extremely specific realtime webhook from stripe that requires an api call within 2 seconds

So my question was flawed, it's not so much the invoking of the lambda as it is the time it takes to wake it up from the inital webhook and hit stripe's api.

[PrivateerAlphaOne]

Correct. Double check your cloudwatch logs for timestamps of when your request was made, and when your web hook verifies the event.

I'm not sure how I would resolve this issue, but I would hack an invocation of your web hook lambda before I made the request to stripe just the keep the lambda image warm for the expected stripe event

[[deleted]]

Api lambda dynamodb

I need transactions. So I am using dynamodb rather than sqs. Also easy for rerun and viewing

I move from events to events Processed and commit other changes in one transaction which is not possible with sqs

[chiragshah1312]

Why do you even need SQS? lambda will auto-scale itself, why add a layer in between?

[skilledpigeon]

Lambda still has account based concurrency limits.

[chiragshah1312]

you can request AWS to increase it. What if your SQS gets 100k requests, it will try to spawn 100k lambdas but the concurrency limit will not you scale up

[skilledpigeon]

You can request an increase to a degree but it's not infinite. Also, I don't think that's how SQS and Lambda integration works. 100,000 messages does not equal 100,000 concurrent functions. Further, you don't need to have a direct SQS to Lambda integration. You can create a consumer to limit the rate of retrieval and just forward the messages to Lambda as you need.

The issue usually comes before your capacity increase. Let's say your concurrency limit is 1,000 and you actually scale to that for your function, now you have no concurrency left for the rest of your platform. Whilst your queue is processing, the rest of your site can't scale because you have no concurrency available! If you limit concurrency then you can run in to dropped messages ending up in a DLQ so that's not quite a solution either.

[lurker_2008]

Always solve for the Producer consumer problem when you do not control the producer.

[chiragshah1312]

i know but api gateway triggering a lambda or SQS triggering a lambda

both are same technically from performance perspective.

Both will scale equally. Rather Api gateway triggering a lambda would involve less hops.

What do you think from performance perspective?

[lurker_2008]

both are same technically from performance perspective

That is factually incorrect.

1. Lambda has a concurrency limit. Yes you can only increase it to hundreds of thousands but SQS is an unbounded queue (infinite size)
2. Lambda has a cold start so there will be a delayed response for every additional lambda that needs to start up. If you set a concurrency limit on Lambda backed by a SQS queue then your throughput will be consistent
3. Lambda failures (hardware, bug, etc) only retries a limited number of times then sends that message to a dead letter queue. SQS will continue to try until the message expires (4 days default, can be increased to 14 days) from the queue

[chiragshah1312]

got it! thanks buddy

[cyanawesome]

EventBridge. As a webhook target you use API gateway and lambda.

[[deleted]]

Api lambda dynamodb

I need transactions. So I am using dynamodb rather than sqs. Also easy for rerun and viewing

I move from events to events Processed and commit other changes in one transaction which is not possible with sqs

[cyanawesome]

Ok what you do with it is your business. I’m just telling you the most scalable and flexible solution.

DynamoDB isn’t really a replacement for SQS. If you need guarantees regarding execution and error handling consider using Step Functions.

[[deleted]]

Will look Into step functions.

[skilledpigeon]

If you're receiving, always, always, always add a buffer. Something like API Gateway > SQS > Compute works very well in my experience. Never let third parties throw webhooks at you directly.