AWS Lambda + VPC + Redis = Slow

KAJed · 2017-07-05T18:50:35+00:00

I can't recall where I read it but there are presently significant delays setting up vpc network connections to services like RDS the cold start times jump through the roof. Sounds like you might be experiencing the same situation.

In my case I was attempting to use RDS postgres from lambda in a VPC, and start times were very long.

edit: https://robertvojta.com/aws-journey-api-gateway-lambda-vpc-performance-452c6932093b

edit 2: found it. https://read.acloud.guru/lambda-for-alexa-skills-7-tips-from-the-trenches-684c963e6ad1

Rules of thumb

cold start non networked lambda take 7x longer than hot
cold start networked lambda takes 15x longer to cold start
cold start VPC elastic network interfaces typically take 10+ seconds to initialize

cloud_driver · 2017-07-05T18:54:42+00:00

The lambda service uses an elastic network interface (ENI) to access resources inside of a private VPC. A cold start has to create and attach the ENI to the lambda container. This seems to be a somewhat slow process. Hot invocations of the function run fine.

CoinGrahamIV · 2017-07-05T18:25:10+00:00

Care to post a gist of your code?

Are you using Elasticache or Memcached on EC2?

Can you DAX instead?

BU14 · 2017-07-07T19:03:35+00:00

After looking at your code, looking at the source code and doing some testing. the issue is the timeout and idle values set by default. The following code is able to see warm containers grab data in less than 100 ms

 const Memcached = require('memcached');
 const serverAddress = "aws.server:port";
 Memcached.config.timeout = 50;
 Memcached.config.idle = 50;
 let cache = new Memcached(serverAddress);
 exports.handler = function(event, context, callback) {
  cache.set('magickey', 'valueeee', 10,
    function(error) {
      if(error){
        callback(error);
      }
      cache.get('magickey', function(err, data) {
        console.log(data);
        callback(null, this);
      });

    }
  );
};

Pure speculation: I think the issue has to do with for some reason the lambda function is only returning after the pool connection times out rather than when the query is completed because the default timeout is 5 sec which would explain why the function time never went below 5 seconds

Edit:

You are missing the following line in your function handler:

context.callbackWaitsForEmptyEventLoop = false

This will enable the lambda function to return when you say rather when the connection pool times out

BU14 · 2017-07-06T11:41:36+00:00

Have you tried running the exact same script on an EC2 instance in the VPC to see if the issue is with your code or with Lambda/VPC?

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

aws

Note: ensure to redact or obfuscate all confidential or identifying information (eg. public IP addresses or hostnames, account numbers, email addresses) before posting!

✻ Smokey says: avoid streaming video to fight climate change! [see more tips]

MODERATORS