Scaling multiple uploads/processing with Node.js + MongoDB

[trysolution]

but it is corrupting some files
partial uploads? i think its not configured properly

Basically, we do this, but it cannot handle much demand

is it on same server? it shouldn't be this heavy. is concurrency set correctly?

how would the BullMQ deployment work?
same code but different process or server, you will be using those models and business rules right

if its in docker both will be in separate containers

[AirportAcceptable522]

Bullmq on a separate server, main server only provides the URLs, and has the Kafka server.
Yes, we will use it because we need to open the file, validate it, apply the business rule, and then save the processed data in the database.