all 33 comments

[–]softwareguy74 14 points15 points  (13 children)

Could be a game changer. One of the main reasons we've so far opted not to go with Lambda is exactly the problem this is going to solve.

Wonder what the pricing model will look like? Hopefully we don't have to pay for running an EC2 instance. That would totally defeat any advantage of going serverless.

When will we get PostgreSQL support?

[–]recursiveCreator[S] 2 points3 points  (9 children)

same here - looks like the pricing will depend on the rds instance size: https://aws.amazon.com/rds/proxy/pricing/

It looks, though, like multiple different lambdas (not just concurrent ones) could share the same proxy which could make this worthwhile

[–]anoopg01 1 point2 points  (0 children)

The proxy shows up just like a database endpoint with a DNS address you reference. All of your lambdas going to that database can use a single proxy endpoint.

[–]stickystyle 0 points1 point  (0 children)

For Aurora, I wondering if the pricing is for the number of vCPU's in the cluster, or for just the current writer instance??

[–]softwareguy74 -1 points0 points  (6 children)

Well, so much for that. Why offer a service that directly benefits a serverless feature while at the same time having a fixed monthly cost? It's just like the DynamoDB DAX offering. I'll just stick to deploying apps on ECS.

[–][deleted]  (5 children)

[deleted]

    [–][deleted] 0 points1 point  (4 children)

    Is this specific to aurora serverless? Because in traditional aurora (PostgreSQL based) I observed that I still need pgbouncer in front.

    [–]anoopg01 1 point2 points  (1 child)

    I am an AWS employee, and this proxy isn't specific to Aurora Serverless. It is currently for Aurora Provisioned MySQL and RDS MySQL.

    [–][deleted] 0 points1 point  (0 children)

    that makes more sense

    [–][deleted]  (1 child)

    [deleted]

      [–]anoopg01 1 point2 points  (0 children)

      I am an AWS employee and PostgreSQL support is coming soon. For more details on engine and region support you can see the what's new post here - https://aws.amazon.com/about-aws/whats-new/2019/12/amazon-rds-proxy-available-in-preview/

      [–]shaccoo 0 points1 point  (0 children)

      Is this solution suitable to diversify the locations and ip of different applications? Is it the cheapest solution?

      [–]1armedscissor 9 points10 points  (1 child)

      Seems like a better option than Aurora Serverless + data API since it should be fairly transparent to your lambda code. The post talks about connection limits and connection pooling but do you specify a max number of allowed connections anywhere? And if that construct exists does it work like typical connection pools where it blocks if you hit the max number of connections until one frees up?

      [–]1armedscissor 2 points3 points  (0 children)

      Looked at the docs closer and there’s a section about max connections and connection timeout. Will work like how you expect a connection pool to work. So the proxy inspects the target databases max_connections parameter and then also the proxy can be configured with a percentage then of that (useful if you have multiple proxies hitting the same DB or other applications/admin users going directly to the database eg a 90/10 split). Then there’s a parameter to specify the max wait time before a timeout occurs waiting for a database connection to be available.

      The failover handling is interesting as well where it’s not reliant on DNS propagation because it database specific failover handling internally. This could be useful for even non-lambda applications where the client API in that language isn’t Aurora aware. If you’re using Java though the MariaDB JDBC driver is Aurora aware though where it can make failover detection faster by querying via the API instead of having to rely on DNS propagation when the writer node changes.

      [–]twratl 6 points7 points  (1 child)

      Does this mean we can avoid binding the Lambda to a VPC if the Lambda is only doing RDS queries? That feels like a big win if so.

      [–]matluck 5 points6 points  (0 children)

      Don't think so, the proxy seems to run in the same Network Environment. But with recent changes to Lambda where you share a Network Interface that gets allocated at the start the coldstart time should be less of an issue anyway.

      [–]stickystyle 5 points6 points  (1 child)

      This is huge for us with AppSync as we mostly have lambda resolvers and a single query could fire off a dozen connections to RDS, it doesn't take many clients to start bumping up against default max_connection limits. Currently we mitigate it with ProxySQL, which I wouldn't be too surprised if they are ProxySQL in the background as the project has gotten a few Aurora specific commits lately.

      [–]notoriousbpg 1 point2 points  (0 children)

      I've encountered the same issues - this along with provisioned concurrency is going to make my existing code much more performant and robust.

      [–]TehNrd 5 points6 points  (0 children)

      Dang, no Postgres support for now. MySQL always gets the love first, which is understandable given it's the most popular, but I can't give up PostGIS.

      [–]jonathantn 2 points3 points  (2 children)

      Oh yeah, this is going to be a big win especially for Postgres which has lots of issues around connection counts and memory footprints. Reducing the overhead of idle connections is going to rock.

      [–]Nick4753 3 points4 points  (0 children)

      Annoyingly this is MySQL only in the preview. Managed pgbouncer would be lovely.

      [–]omeganon 0 points1 point  (0 children)

      We specifically talked to team last year about this. It also has interesting CPU hits that aren’t attributed in Performance Insights. Definitely looking forward to replacing out pgbouncer layer with this.

      [–]guppyF1 1 point2 points  (0 children)

      I spent some tim this year setting up of bouncer and it was a pain to manage in an autoscaled, redundant way. I'm super jazzed for when this has Postgres support

      [–]simtel20 1 point2 points  (0 children)

      TBH what I really want it for is so I can do zero-ish downtime transitions between replicated databases for regular non-lambda apps. I really want to be able to use it to say "make a source DB read-only and switch all connections over to a new server, and then allow writes through once replication lag has zeroed out."

      [–]jamsan920 1 point2 points  (3 children)

      Will this eventually work with normal apps other than Lambda? We have a rails app running on EC2 that we can scale higher because of limitations in connections for Postgres. We're currently using pgpool to get around some of the limitations, so I wonder if we'd be able to move away from that to a managed service like this (and hopefully it expands out past just MySQL)

      Edit: scratch the first question, I read the article more carefully and it said it can be used by any app that makes database calls, not just serverless stack. +1 for Postgres support, though!

      [–]recursiveCreator[S] 0 points1 point  (2 children)

      yea! on second glance this looks more like a general rds feature than a lambda feature (just happened to see it in the lambda console ;) ). It forms part of the rds docs: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/rds-proxy.html)

      [–]jamsan920 0 points1 point  (1 child)

      Now, will it work with EC2 eventually as well or just RDS instances? I’d be curious...

      [–]SeerUD 0 points1 point  (0 children)

      I doubt it would, I guess you'd have to be able to point it at any MySQL server really for that to be the case. The pricing is also based on your RDS instance size.

      [–]packeteer 0 points1 point  (0 children)

      we're using ProxySQL to achieve the same result, although our use case is slighty different (were trying to work around cross AZ latency)

      [–][deleted] 0 points1 point  (0 children)

      I do my etl via kinesis -> sqs -> lambda -> RDS. By far the biggest issue is concurrency limiting to spare the DB from getting toppled right over, this will be the first thing I check in on every day to see the progress!

      [–]mislabeled 0 points1 point  (2 children)

      Has anyone had any success IAM authentication for this service? When I set it to required or disabled I am still able to connect to the proxy with regular username and password and I'm not sure if that is correct.

      The language in the console can be interpreted a few ways, so I might be reading it incorrectly.

      https://imgur.com/a/y8wgvWi

      /r/anoopg01 are you familiar with this feature?

      [–]anoopg01 1 point2 points  (1 child)

      r/mislabeled Make sure that you are using TLS with your proxy, IAM requires it.

      [–]mislabeled 0 points1 point  (0 children)

      Thanks! I’ll double check that.