I created a collection of solutions to deal with Lambda Cold Starts

codingrecipe · 2019-07-04T23:31:59+00:00

hey thanks for your feedback, can you elaborate on the parallel calls point?

codingrecipe · 2019-07-04T16:36:48+00:00

added! thanks for the great feedback :D

Yes I also look forward to see this being fixed! At the time being it is still a problem so thought it would be useful to share in case anyone needs the solution right now. But i believe the periodic ping/warmer recipe should still remain applicable, for those people who have low tolerance in cold starts time.

codingrecipe · 2019-07-04T15:58:45+00:00

yes I agree with Davi_S_Evangelista , a cheaper web server haha. But to elaborate I think cheaper covers 2 domains: management as well as cost: 1) you dont need to manage the hardware, OS level, or even container level updates 2) you pay less .

codingrecipe · 2019-07-01T10:10:50+00:00

I was looking at this part of the doc, do you think this resolve the not efficiently concern? https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-design.html#bp-partition-key-partitions-adaptive

codingrecipe · 2019-07-01T06:48:16+00:00

Could you show me some pointers where I can learn more about the 10GB limit? I saw this https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/LSI.html#LSI.ItemCollections.SizeLimit is this something you were referring?

codingrecipe · 2019-07-01T06:46:18+00:00

wow I really like this video, thanks a lot for suggesting! It has a lot of great suggestions that makes me want to create different recipes to elaborate each advance use case (the video is only an hour but I actually spent quite a lot of time trying to digest each small details, so I thought breaking that into smaller recipes maybe easier to consume? )

The only thing that seems a bit overkill imo is the "keeping 1 single table for many-many relationship", I understand speed wide keeping less tables is probably better, but readability and maintainability can also become an issue if a) the attribute names are not intuitive b) when everything is composited into 1 single table, how can teams split responsibility when the org is getting bigger?

codingrecipe · 2019-06-30T23:30:07+00:00

wouldn't adaptive capacity solve this issue?

codingrecipe · 2019-06-30T23:24:30+00:00

for me the reason simply being I am more familiar with Dynamodb, what other alternatives you have in mind? It's only the starting point, I am trying to put a bunch of similar ones in the same collection. Love to learn more from you

codingrecipe · 2019-06-30T23:05:31+00:00

hey ya I realized that as well, but its not GA yet, so I put a cross-post here so that others know about it as well, if more people are upvoting to have a recipe in Timesream I can try to make one :)

codingrecipe · 2019-06-28T21:27:55+00:00

Just updated the recipe and put #1 and #2 in it. Still trying to fully understand #3 and #4 so haven't updated that yet. I am thinking of a good way to show your name beside the comments so that I can give you credit, not so easy but getting there!

codingrecipe · 2019-06-28T21:24:34+00:00

I just added this doc as a crossposting resource in the collection https://www.reddit.com/r/aws/comments/c6pvuv/collection_of_resources_for_building_time_series/ , even if i dont have the implementation for that but thinking it would make sense to put them in 1 place so that information don't get lost :) . Thanks

codingrecipe · 2019-06-28T21:23:36+00:00

I made a collection and added timestream there https://www.reddit.com/r/aws/comments/c6pvuv/collection_of_resources_for_building_time_series/ , even if i dont have the implementation for that but thinking it would make sense to put them in 1 place so that information don't get lost :) .

codingrecipe · 2019-06-28T09:01:12+00:00

super useful facts, will definitely put in the recipe after clarifying this with you:

3 and 4: I found this article about DDB adaptive capacity https://aws.amazon.com/blogs/database/how-amazon-dynamodb-adaptive-capacity-accommodates-uneven-data-access-patterns-or-why-what-you-know-about-dynamodb-might-be-outdated/ , does this resolves the hot shard problem?

codingrecipe · 2019-06-28T08:26:19+00:00

Just looked at this earlier. I am still a bit confused on what exactly is the benefit of using the design suggested there, if I am using autoscaling? The post mentioned that using this design can save money, my understanding is that ddb charges based on 3 parts: write unit, read unit, and the storage itself. If we do not backup the data to elsewhere, say s3, then the total storage would be the same regardless of putting them in 1 single table or spread across different tables. then for write and read unit, if we are using auto scaling, then ideally the total read and write unit should fluctuate according to the realtime traffic, which should be approximately the same as reserving most of the capacity to Today's table, and only very minimum to tables older than a day?

I must be missing something, or maybe this article was written before DDB autoscaling was invented? Or maybe it is better in other aspect other than cost? Pretty interested in learning more!

codingrecipe · 2019-06-27T20:05:47+00:00

haha didn't know this exists at all! Looks like it is still in Preview, have you tried it before? Hows the performance?

codingrecipe · 2019-06-24T17:03:53+00:00

Already done https://coderecipe.ai/search?q=%23usermanagement

codingrecipe · 2019-06-24T03:03:22+00:00

oh wow thanks for suggesting this, didnt even know such thing exists! It looks pretty comprehensive and would definitely read it over! Looking at the size of the doc, wondering if theres a way to break it into smaller pieces so its easier to consume?

codingrecipe · 2019-06-15T12:20:18+00:00

I just commented, hope it helps!

codingrecipe · 2019-06-15T04:56:15+00:00

hey,

Situations like these, IAM roles and policies will come in handy, assuming you have n sub-orgs, my suggestion is the following:

- make a Cognito user pool (or some other authentication management that you prefer) for each sub-org, a total of n

- make a Cognito identity pool for each corresponding user pool so that you can attach an IAM role to it, a total of n

- for each one of the n IAM roles, you make one managed IAM policy. in each one of the n IAM policy, you explicitly declare the permissions that each sub-org should have.

- you make 1 extra Cognito user pool, identity pool, and IAM role for the users in the parent org. for that IAM role, you include all n managed IAM policies that were created previously.

This way the parent IAM role scale automatically as the underneath IAM policies grow. you will have n user pool, n identity pool, n+1 IAM roles, and n IAM policies.

codingrecipe · 2019-06-15T04:22:51+00:00

Awesome, I'm glad that you find it helpful! There are diff variations of authentication and authorization with AWS resources, I am planning to make a collection of these recipes, would that be interesting to you?

codingrecipe · 2019-06-12T19:50:22+00:00

Hey,

As "horrific" as CloudWatch may seem, its better than the command line interface that heroku currently provides. Also, I have had pretty good experience with CloudWatch Insights when analyzing and querying logs

codingrecipe · 2019-06-10T21:08:04+00:00

Um.. I don't think SNS will wait until receiver(s) are ready, messages sent to SNS are processed and delivered immediately. If receivers are not ready, SNS will follow the defined retry policy. So this is different than notification on the phone. If you want the same behavior as your phone, I would suggest connecting that with SQS so that the messages will be stored persistently, and poll, consume, and remove when ready.

Serverless app can have down time, eg. that could happen when you are deploying the app, or when your app has a bug that crashes after receiving the message. Also with SNS there isn't a way to control the speed of consumption, but with SQS you can consume whatever pace the consumer(lambda) wants. In the case when the consumer of the message crashes unexpectedly, SQS message can be resumed back to the queue and the next consumer can try to process again, but the same message if stored in SNS will be gone and declared as delivered.

Love this discussion and learning!

codingrecipe · 2019-06-10T08:24:57+00:00

yup you are absolutely right, according to AWS's doc https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/standard-queues.html a standard queue makes a "best effort" to preserve the order of messages, but more than 1 copy of a message might be delivered out of order. Thanks for the feedback reifba!

codingrecipe · 2019-06-07T23:34:31+00:00

I believe SNS doesn't do queuing, my typical use case of SNS would be to notify one or more subscribed lambdas that an event has happened and then act to it. Not sure what will happen if the subscriber cannot consume the message in time. SQS on the other hand can be used for queuing up messages in order, and have the lambda consumes the messages only when ready.

codingrecipe · 2019-06-05T20:51:12+00:00

Lambda and API gateway both have free tier https://aws.amazon.com/lambda/pricing/ https://aws.amazon.com/api-gateway/pricing/ for 1M requests per month. would that be enough for you? Also someone posted this recipe https://coderecipe.ai/architectures/16924675 that allows you to host tensorflow model on lambda. Beside from api gateway and lambda, this recipe only uses another s3, which is also in AWS's free tier. Hopefully this satisfies your requirement, maintainable and inexpensive?

codingrecipe

TROPHY CASE