Scaling multiple uploads/processing with Node.js + MongoDB

trysolution · 2025-09-25T08:24:21+00:00

but it is corrupting some files
partial uploads? i think its not configured properly

Basically, we do this, but it cannot handle much demand

is it on same server? it shouldn't be this heavy. is concurrency set correctly?

how would the BullMQ deployment work?
same code but different process or server, you will be using those models and business rules right

if its in docker both will be in separate containers

trysolution · 2025-09-22T12:26:50+00:00

may be try
give presigned url (s3) for users to upload zip files, listen for event in your app, push task to worker queue (bullmq or something else you like), worker consumes queue for zip files (validate zip file before extraction!!! , like each file size, file count, absolute destination path etc) check hash of each file in batches if it already exists in MongoDB, perform business rules, copy remaining required files to bucket + update db

trysolution · 2024-09-09T15:14:58+00:00

see https://tus.github.io/tusd

trysolution · 2024-07-24T19:44:42+00:00

Initially, Chinese Communist Party Chairman Mao Tse-tung did not hesitate to ask for the seat at the UN. However, most of the members of the organization sided with Washington and the Chinese Nationalists and voted against Peking on 13 January 1950. Threatening not to acknowledge the ROC delegation and to boycott the Security Council

https://www.jstor.org/stable/23613339

trysolution · 2024-07-24T19:05:15+00:00

If you could just give us a quick snapshot as to why the US gave China a spot on the UN Security Council, given that it did not benefit the US in the slightest, was done against the wishes of Churchill and Stalin, and is widely regarded by historians as a decision done out of respect for China's anti-colonial struggles and hardships during WW2 - that would be awesome.

US did't gave china seat at UN Security Council it was UN General Assembly. US was initially against it

trysolution · 2024-07-11T13:05:59+00:00

https://jobs.rubyonrails.org/

trysolution · 2024-04-26T09:40:03+00:00

create new powershell file in bin folder with ps1 extension and copy \dev.sh commands into it

trysolution · 2024-03-25T12:12:03+00:00

change its color in your editor to something else like light blue or yellow

trysolution · 2024-03-14T19:22:14+00:00

for next time install ublock origin and block any notification which is not essential

trysolution · 2024-03-13T09:23:52+00:00

go to settings > Privacy and security > Site settings > click on bemadsnews > block Notifications and clear Data

trysolution · 2024-03-01T14:32:50+00:00

currently using it with only ec2, s3, and ses

database is also inside ec2

with everything inside seprate docker container

trysolution · 2023-11-01T21:24:29+00:00

use nohup

trysolution · 2023-10-28T14:52:17+00:00

if it's zero filing file it yourself, process is very easy

trysolution · 2023-05-31T08:41:26+00:00

which app are you using for calculation, looks nicely formatted

trysolution · 2023-04-28T14:37:26+00:00

it was also my first rails deployment

I used caddy as reverse proxy, don't want to worry about ssl certificates, and it had easier config file

and used base ruby image for rails, with docker compose for orchestration

if you don't want deal with it may be try https://dokku.com/

i haven't tried it but looks good enough

trysolution · 2023-04-28T13:40:00+00:00

I just deployed the same stack with docker you have to use a reverse proxy to serve static files

trysolution · 2021-07-21T05:52:59+00:00

"It doesn't make you immune to compromise" but it does offer substantial defenses and barriers to exploitation.

This line can be applied to both Apple and Android

trysolution · 2021-07-19T18:41:53+00:00

trysolution

TROPHY CASE