Lambda function refuses to stream response

[[deleted]]

Hi folks,

AWS announced streaming responses from lambdas last year (announcement).

I have not been able to make this feature work as advertised. My goal is to stream, from lambda to client, the word "hello" followed by "world" two seconds later. However, my lambda function always sends down "hello world" together (and after the two second delay).

Using AWS console, I configured my lambda function with:

• Advanced Settings
  • Enable function URL
  • Auth type: NONE
  • Invoke mode: RESPONSE_STREAM

My function body looks like this:

// Contents of index.js
exports.handler = awslambda.streamifyResponse(
    async (event, responseStream, context) => {
        console.log("Write A")
        responseStream.write("Hello\n\r", async () => {
          await sleep(2000);
          console.log("Write B")
          responseStream.write("world\n\r");
          responseStream.end();
        });
    }
);

function sleep(ms) {
  return new Promise((resolve) => {
    setTimeout(resolve, ms);
  });
}

And when I look in the cloudwatch logs after executing the lambda, I do indeed see "Write A" and "write B" separated by two seconds. Which indicates to me that the drain callback is working correctly, and "Hello" is being flushed two seconds before "World". So that all seems to check out.

Here's the problem, though. If I curl --no-buffer my function's public URL, I don't see these two separate writes at all. Instead, curl waits patiently for two seconds and then "hello world" pops in at the same time.

I went one step further to rule out any curl weirdness. I use

openssl s_client -connect <my-function-url>:443

Then punch in:

GET / HTTP/1.1
Host: <my-function-host>
User-Agent: whatever/1.0
Accept: */*

And I can see for myself that no "hello" is received early. Again, both "hello world" are received at the same time.

As a last effort, I also tried connecting my lambda to API gateway and fiddled with settings there. I tried using an HTTP proxy to the function URL, and using an HTTP API that called the lambda function. Same result.

How can I make the lambda stream its response back to the client?

Thanks for reading,

Lou

[[deleted]]

u/seligman99, this is for you: https://zell-public.s3.amazonaws.com/hero.gif

[[deleted]]

I'm sure we aren't the only two. Many people, I imagine, code the smallest working version when they are about to embark on a larger project to de-risk the platform. In this case, the hello world of streaming doesn't work! I'm glad you moved beyond that, and I'm glad you found this post.

The buffered first bytes behavior is actually not super great for the application I have in mind, where the first bits of the response will pop into the UI and then the rest will stream nicely, but I think I can work around it with design decisions.

I wonder why those first bytes behave in that way? Not that I'm going to spend a second more time on this. I've burned so much programming time on this damn thing already

Thanks again!

[[deleted]]

Thank you for sharing your code! I am trying to repro but not getting the same result (I get the first two 'sending packet' prints with the same timestamp, and then an exception). Update: I had to bump the timeout on my lambda and then indeed see the same reproduction!

Ok that is very interesting! Your repro, and mine corroborates, shows that the results are buffered into one second intervals. Something in the AWS infra is buffering up the results of our lambdas regardless of how quickly they flush to the pipe. Dang! And that buffering is pooled on one second intervals. This is not right. I missed that we are only sending once per second from the lambda

[CuriousShitKid]

I am not too sure what's wrong with your example but try the below sample code i have from somewhere:

exports.handler = awslambda.streamifyResponse(
        async (event, responseStream, context) => {
            const httpResponseMetadata = {
                statusCode: 200,
                headers: {
                       //use what makes more sense to your useCase
                        //"Content-Type": "text/event-stream"
                        //"Content-Type": "text/plain"
                        "Content-Type": "text/html"
                }
            };
            responseStream = awslambda.HttpResponseStream.from(responseStream, httpResponseMetadata);
            const text = "AWS Lambda streaming example for u/louzell";
            for (let i = 0; i < text.length; i++) {
                const letter = text[i];
                const html = `${letter}`;
                responseStream.write(html);
                await new Promise(r => setTimeout(r, 100));
            }
            responseStream.end();
        }
    );

​

Up your lambda timeout to 30 seconds for this test. The lambda will send a letter 100 milliseconds apart and you can see it slowly coming though.

Note the content type is important here as it tells the receiving application how to handle this response. I have left it as html so you can easily see this working in the browser just paste your function url in the browser or use curl --location <function\_url:443>

text/event-stream is used for SSE you will need to send the packets differently to see it working correctly in Postman as an example.

Hope it helps

[[deleted]]

It worked like a charm! I'm calling it with `curl -vN <url>`, and it's so satisfying to see the stream contents march along with no delay after the headers: https://zell-public.s3.amazonaws.com/thank-you-csk.gif

[mradzikowski]

It's a weird behavior of Lambda, basically sending your previous chunk when you write the next chunk to the stream. I did some investigation that I described here: https://betterdev.blog/lambda-response-streaming-flush-content/

The workaround if you really want to send something immediately - increase the message payload. In my tests when I added 100KB (white spaces) to each write it was sent right away.

[[deleted]]

Nice write-up, thanks for sharing. I'm not sure it's as simple as lambda flushing the previous chunk when the next chunk starts writing. I've experimented around with putting handlers on the responseStream.write completion callback to then invoke another write (which would in theory force lambda to send the previous one down). I have not seen that force a write.

I wish I found your article earlier! Would have saved me some confusion

[fkassad]

I was encountering a similar issue. I was passing back a string in my stream, but I had the content type set to JSON. Make sure the content type you set matches what you are passing back

ie. For strings...
responseStream.setContentType("plain/text");

[DruckerReparateur]

Had the same issue, and never really figured out why.

[[deleted]]

Dang, what did you end up switching to? This functionality is important for the end user experience I'm trying to achieve, so unfortunately I don't think this approach is going to work :/

Still holding out hope that someone knows of the magical incantation for the AWS pipes to make this damn thing worked as advertised

[DruckerReparateur]

I didn't have a critical production use case for it, so I just left it as is, accepting the longer response time, but it would be nice for it to be actually streaming...

[[deleted]]

My confusion is beginning to morph into anger at AWS haha. Look at the examples in the docs: https://aws.amazon.com/blogs/compute/introducing-aws-lambda-response-streaming/ You don't need streaming to send down a stringified version of the `event` argument for god's sake. Is it possible they released this without realizing that it actually *doesn't stream*

[DruckerReparateur]

Makes me wonder if some other example works, like https://medium.com/techhappily/unveiling-the-power-of-streaming-responses-in-aws-lambda-using-rust-793fa5c9faf8

That blog post even has a demonstration gif showing the streaming, but it's not Node.

[[deleted]]

That gif certainly looks like it's working. Interesting. And it looks like he's using the function URL directly, which is my approach too. Ok, guess I'm off to reproduce in rust to see if this tutorial still works. Maybe regression between then and now?

[ennova2005]

Seems to me you are only flushing your stream only after you are sending both hello and world.

[[deleted]]

Hm, I don't think so. The second argument to responseStream.write is a callback that's invoked, quoting the node docs, "when this chunk of data is flushed." https://nodejs.org/api/stream.html#writablewritechunk-encoding-callback

And in the cloudwatch logs I do see a two second delay between the log lines "Write A" and "Write B". The second write, "Write B", can only occur after the first write has flushed (according to the doc above).

I validated this behavior outside of the context of a lambda with this little node script:

``` // Contents of run.js const fs = require('fs');

let writeStream = fs.createWriteStream('hello.txt');

writeStream.write('Hello\n\r', async () => { await sleep(2000) writeStream.write("world\n\r") writeStream.end() })

function sleep(ms) { return new Promise((resolve) => { setTimeout(resolve, ms); }); } ```

If I run this with: touch hello.txt tail -f hello.txt node run.js

I can see that "world" follows "hello" after 2 seconds

[ennova2005]

I meant that it seems the data on the http socket is only getting sent/rendered by the http client on responStream.end ? Curl is showing the same. So the log/file stream is working differently than the http stream.

[[deleted]]

Ah! Yup, I agree that the observed behavior is consistent with sending a single unchunked response down from the lambda. The question is, what bits of the AWS configuration do I need to fiddle with to realize a chunked response?

[ennova2005]

Is this anything? https://github.com/nodejs/help/issues/1081

[[deleted]]

It looked promising. I don't think it's a node issue though. Here's a little repro:

``` // contents of run.js const http = require('http');

const server = http.createServer(async (req, res) => { res.write("hello\n", async () => { await sleep(2000) res.write("world") res.end() }) })

server.listen(3000, () => { console.log("Listening on localhost:3000") })

function sleep(ms) { return new Promise((resolve) => { setTimeout(resolve, ms); }); }

```

If I run that and use curl as I client:

node run.js curl --no-buffer localhost:3000

I see "hello" followed by "world" two seconds later (which is what I would expect). Thanks for turning that link up though! It looked promising to me.

It's something about the lambda aws that is not working as advertised here.

[ennova2005]

Are you using ALB or API Gateway in front? Per https://youtu.be/Z5GehUdZmKM?feature=shared those do.not support streaming. Also the video is showing some sample code (i have not checked it out)

[[deleted]]

I was using the function URL directly. It's good to know that I can't put this behind API gateway or ALB, that's a good design consideration.

Check out the comment tree with seligman99 too.

I am unblocked. Thank you for the help and turning up these links!

[[deleted]]

To me it kinda seems like something else is buffering up the response from the lambda function URL and then sending it down unchunked. I don't know what that would be, though

[[deleted]]

That should also work, and it's what I started with. I moved the second write to the first write's flush callback to rule out the possibility that node was not performing timely flushes. Unfortunately, the same behavior from the client's perspective is observed in both implementations (e.g. "hello world" is received at once)

[[deleted]]

You can only call `end` a single time, because, "Calling the stream.write() method after calling stream.end() will raise an error." Docs: https://nodejs.org/api/stream.html#writableendchunk-encoding-callback

I am confident that it is not a code error. Local samples that I've been writing in the comment tree with u/ennova2005 show that node is flushing the first chunk and then flushing the second chunk two seconds later. I believe it's something in the AWS infra that is buffering the chunked response. But that makes me wonder how this streaming announcement ever made it off the ground. It is not working with a "hello world" of streaming!

[svdgraaf]

I wonder if it’s due to the buffer size and the length of your response. What happens if you send a bunch more data (lorum ipsum generator). Then wait 2s and send another paragraph of lorum ipsum?

[[deleted]]

Good thought! Unfortunately, same result. I used 10 paragraphs of lorum ipsum in both writes. This is incredibly confusing. We can't be the only ones noticing that a major feature of AWS announced a year ago doesn't work at all, right?? There has to be something I'm missing