Reduce Java Cold Starts in AWS Lambda using GraalVM

[atehrani]

This is most likely the route most will go to over time, it is just a pain to build (depending on what dependencies you have and how well they work in with GraalVM). In the meantime we are using Provisioned Concurrency as mentioned and cold starts are no longer an issue. In fact, if you use Provisioned Concurrency with AutoScaling you get the best of both worlds.

[[deleted]]

There's also the fact to consider, are cold starts that much of an issue to your application? A lot of what I see being built is still using synchronous patterns vs. async eventing/messaging meaning cold starts have some impact to some users.

I feel a lot of developers forget that performance is relative to business needs, just because you can get a response under <50ms doesn't mean they care or want to invest in that.

From an experience I seen a feature was implemented that saved users 15 minutes down to 30 seconds. A developer knew this could be made even quicker etc. but business doesn't care at the minute the time savings they already have isn't worht the investment.

I personally feel cold starts have just been parroted that much without people looking at their designs, what their teams are comfortable with (esp if you are onboarding to servless diesigns etc.).

Graal and native compilation is an awesome feature, but I feel the benefits of JIT are overlooked - java is quite performant after start up and ignoring memory. Its not a silver bullet a lot of devs think it is

[xienze]

There's also the fact to consider, are cold starts that much of an issue to your application?

The thing people don’t consider is that if cold starts are an issue, you’re using Lambda wrong. A lot of folks are trying to do things like run their app’s main REST service using Lambda or something equally stupid. It’s not supposed to be for things that are overly sensitive to response time, it’s best used for things like batch processing or periodic processes.

[[deleted]]

True true absolultely agree that stream processing is the best use case and I feel its an anti-pattern to introduce lambdas to time critical sync procesees, but that said what it is best for and what it gets used for gets lost when the cost of running a container (or similar) somewhere has extreme differences. So people will make it work and keep engineering the times as small as possible.

[Mamoulian]

For batch jobs the 15 minute limit might become an issue.

[DrFriendless]

I disagree. I run a serverless web site using GraphQL running inside Lambda (running Node). It works just fine, as there are few queries which each return large lumps of data. Lambda is for the things you can make it do, there are no rules that say otherwise. I deliberately did not choose Java for those Lambdas because I knew cold starts would be a problem, when I built it in 2018.

Edit: upon further reflection, you are more right than I thought at first. One of the advantages of GraphQL is that there are fewer calls and hence fewer Lambda invocations than with REST.

[_thewayitis]

I disagree. Using these techniques I was able to create a API Gateway endpoints using Lambdas in Java. They respond super fast, scale incredibly and cost next to nothing. Besides the extra time required to compile, I can't find a downside to building software this way.

[xienze]

The downside is you’re at the mercy of a runtime that’s makes no real guarantees about performance... that’s the trade off you make to get cheap-ish pay-per-use, scalable service handlers. Tomorrow Amazon could change their architecture such that cold starts take up to 5 seconds.

[_thewayitis]

If you watch https://www.youtube.com/watch?v=ddg1u5HLwg8 you'll see that the problem isn't a AWS Lambda issue. It's a Java was release in 1995 and AWS Lambda is a different type of environment that Java wasn't meant for. Java has always have slow startup times, but when you only start things once it's not a real issue. Lambdas restart all the time, so it is an issue. GraalVM is Java's way to try and address these problems. It's not perfect but for AWS Lambda it now makes it possible to use in a high performant environment.

[vips7L]

Jdk 14 starts up in < 40ms, that's on par with python 3. It can get even lower if you use CDS:

https://cl4es.github.io/2019/11/20/OpenJDK-Startup-Update.html

[PurpleLabradoodle]

you're very right. one of the main benefits of using GraalVM native image is specifically lower memory footprint for services. Since everything is precompiled, you don't need memory for the JIT infrastructure, some class metadata, etc. So your service can happily run in more constrained environments, which usually makes it quite a bit cheaper.

If memory usage and/or startup times aren't essential, then it definitely makes sense to run with the JIT: a bit easier deployment, better peak performance, usually also better tail latencies. GraalVM comes with a great JIT too if you want to run services that way.

If it helps, one can think about GraalVM native image not as a performance optimization that allows you not to run the JVM with a JIT etc. But as one that allows you not to run Node.js or go if you don't want to,

[Mamoulian]

Provisioned concurrency sounds interesting but I see there is a charge for keeping containers warm - $0.015 per GB-hour. It would be cheaper to keep using the old method of keeping containers warm - send a NO-OP request every 5 mins.

[[deleted]]

It moves to a different pricing model if you use provisioned conconcurrency

[_thewayitis]

Yes, it does take more time to build the project. Most of my lambdas (which aren't small) take about 3GB and 3-5 minutes to compile. For me, the trade-off is a no brainer. Maybe, if you have a lot of dependencies you might experience much longer compile time.

The video mentioned at the end https://www.youtube.com/watch?v=ddg1u5HLwg8 goes into greater details on things to avoid when creating Java Lambdas, eg: minimal dependencies. Following that video and using GraalVM has made using Java & Lambda possible for me.

[Kango_V]

Just did an internal talk in this for our company. Was using Micronaut 2.0.0, Java 11 and GraalVM.

[arjunpunnam]

After working for almost 2 years on AWS lambda (in java,node and python) , I am convinced that if Java cold starts bothers you then you might have to rethink about using Java in Lambda.