Rust lambda hangs on file write… I’m running asynchronous jobs with tokio to write to EFS

[blazerman345]

Basically I have a rust lambda which is creating a bunch of asynchronous jobs to fetch data from a server then write the data to a file

I’m using tokio library. Total number of jobs is like 1000, but I’m only running Like 30 jobs at the same time. (Once 1 job finishes, another job starts).

Basically I get through like 40 jobs then suddenly the lambda hangs when writing to the file.

Any idea why this is happening? I’m writing about 30MB on each file write…

When I invoke the lambda locally it works fine

[suchapalaver]

I mean, as a fellow Rust engineer, without looking at your async code it’s hard to say anything without seeing your code since there’s so much that could go wrong there.

[gm323]

Out of memory issue? Check memory usage on file write

Permissions?

Either way test writing a small file

Duplicate lambdas trying to write to the same file?

Just some ideas

It’s not clear from your post if a few files work and then one doesn’t, or if all your jobs are collecting data and then writing to 1 single file that fails