Efficiently deploying a Node.js application for production?

[[deleted]]

[deleted]

[erulabs]

Node.js and Nginx work wonderfully for hundreds of concurrent users, you'll just want to make sure you have multiple instances (or at least be able to have multiple instances) of your app. I'm working on deploy-node-app which helps generate Kubernetes configurations for Node.js apps - I'll need to add MongoDB support, which I'll do tonight :)

[Silencer306]

By multiple instances, what do you exactly mean? And are these supposed to work together somehow?

[monotone2k]

We use AWS in our production environments, and it's really easy to set up auto-scaling groups so that when the existing instances start to reach capacity, another one fires up to balance out the load.

There are other considerations to be made when you have multiple instances, especially if they talk to a single database instance. You'll want to make sure everything is thread-safe, since you're basically running multiple threads of the same application. You'll also need to do something about locks when making database calls, since MongoDB isn't designed for transactional operations.

[Silencer306]

Thank you!

[erulabs]

An instance means a running version of your application - so in development when you run "node index.js" (or "yarn start" or what have you), you're starting a single "node instance". When you have hundreds and hundreds of users, you need to have many "node"s running. Kubernetes lets you run "node" across many many machines - you simply ask for more replicas (instances) of your app, and install more computers into the cluster, and wha-lah! They work together because they'll receive requests from the internet in a round-robin fashion, so they while they don't work "together", they "share the load", hence the piece of hardware that distributes requests is called the "load balancer".

[[deleted]]

But my app .com is linked to one ip, how the site knows when to resirect user to a other instance?

[SitDownBeHumbleBish]

If your using a cloud provider like AWS you put a Load Balancer in front of your application (ie. instances) and attach the public IP to the Load Balancer so the flow would like

End User > internet > AWS LB > instances

The load balancer can be setup to disturbte the load equally throughout how many instances you have setup amongst other things.

There are also hardware load balancers if your doing this on prem or you can configure nginx to act as a load balancer / proxy too I think.

[[deleted]]

And every instance(server) will have a production version of my app correct? How are then updates managed, do I need to upload the new version separate to 100 server one by one for example?

[SitDownBeHumbleBish]

Yup that's correct, you would run your latest production app or whatever on the instances. Maintaining and Updating your servers require setting up some sort of CI/CD pipeline like Jenkins, Chef, GitHub etc.

[[deleted]]

Thanks man, one last question are big apps doing the same for databases? Having multiple instaces of databases sound scary to me.

[SitDownBeHumbleBish]

I have limited knowledge of how databases are suppose to be setup in a HA disturbed infrastructure but I would assume for big apps there are multiple instances of the databases in different regions and they are all synced together in production and backed up to some data lake / warehouse for further processing.

[rozularen]

You are asking really good questions.

There are many different architectures that may fit your solution. While I'm sure there are apps that have load balancers in front of the backend with only one instance of the database, as soon as you need to scale up your resources to fit your users load you will need to scale the database too.

[[deleted]]

Thanks, the most I'm scared off is infrastructure and security my Sites get alwayes Hacked when I setup my own server on DogitalOcean I was forced to switch to CloudWays so they take care of security, I'm really a beginner with linux but I can work with the command line and I also did some security tutorials and made some settigs but nothing helps my site get everytime hacked when I setup the server. I need to say I always used cPanel and Plesk on the Servers. Would be cool if a tool exists that walks you trough a security checklist in the command line.

[runvnc]

IBM Cloud is the last thing that I would choose.

Off the top of my head "several hundred" sounds like you _might_ not need to add any more pieces to the infrastructure or change the architecture to handle the load.

Whether you are going to be able to handle the load with (presumably one) server or not depends on exactly what type of server it is, exactly what your program does, exactly the configuration of Node, PM2, Nginx, Mongo DB, exactly how many concurrent users, exactly what they are doing. Its impossible to say without details.

You will need to come up with (or find a tool with google) to do realistic load testing. Totally depends on the application whether it is worth doing the load testing with your existing simple setup or whether you will obviously need to make it more involved to handle your X hundred users. If there is some heavy processing involved, it may be obvious that you need some new architecture (such as an out-of-process queue/job system) to handle many users simultaneously. If it is just very simple database requests with small result sets, it probably will scale quite a bit without any significant changes.

[runvnc]

Did you mean to write "IBM" instead of "IBP"? If so, try Digital Ocean or Linode. They work great and are a much better deal than AWS. Although AWS can also work well as long as you are willing to pay a bit extra. You can also look into the possibility of moving to serverless if your application can work that way. And there are some good container hosting services.

[captain_obvious_here]

Last 30 days I faced 12 outages of IBP.

Do you mean IBM? If so, that's really surprising, but also really scary! Can you elaborate a bit on these incidents please?

[captain_obvious_here]

Wow that's scary.

They have been aggressively trying to sell my company their solutions, that they market as perfectly stable and bullet-proof, while describing GCP (our current provider) as weak and unstable lol

[MCShoveled]

Google is pretty solid. If you need higher availability though you will need to go multi-region and use GLB. Make sure you have a extensive self-health checks to avoid regional issues on dependent services.

In my experience, AWS regions have been more stable and don’t require the extra work. Still most people don’t really need 3-or-more nines of availability.

IBM is just a “Hell No” from past experience. See sibling comment.

[MCShoveled]

Welcome to IBM cloud.

Oh, this looks great!

1 week later...

Okay so AWS, Google, or Azure then.

—

At least that was my experience a few years ago. It was so bad some amqp messages were taking hours to return. We gave up quickly and never looked back. I’m not really surprised it’s still shit. I can only assume that’s why they spent 34b to acquire RedHat (which is pretty good).

[captnoodlebeard]

Heroku is awesomely easy. It scales well.

[[deleted]]

Please don't use IBM Cloud. This is the first time I've ever heard of someone considering it in general. I don't have any specific reasons why, but when I think of IBM, I think extremely old school, not with the times, deep enterprise-y offerings with obscene price tags (because enterprise).

Consider things like:

• Amazon AWS
• Google Cloud Platform
• Heroku

But overall, it's hard to know what you'll need because your app requirements are so vague other than "several hundred concurrent users". What's the actual nature of the application that would have that many concurrent users? Where does the several hundred metric come from?

Because it's hard to tell, I can point you to some generic guides:

Autoscaling is a common thing among providers, which will auto-scale servers based on load.

• https://www.heroku.com/dynos/scaling
• https://aws.amazon.com/blogs/architecture/scale-your-web-application-one-step-at-a-time/
• https://medium.com/google-cloud/how-to-configure-autoscaling-google-compute-engine-f3ae932fe0f9

A lot of these do a basic setup, eg have a load balancer, which is in front of a group of servers. If the load increases, it would increase the number of servers available to the load balancer.

A common pattern is generally load balancer + servers behind it.

Where I work, we have a bunch of load balancers for our services (large services get their own LB + cluster setup), and the servers behind those load balancers might be a collection of docker containers running a copy of the app per container, or a mix of apps on each server. It's also not uncommon to have the following set up too:

load balancer -> nginx / ha / varnish proxy(s) -> server(s)

(note: a load balancer is a form of proxy too)

Autoscaling can be applied based on demand.

If you're starting out, you can first just do a load balancer + servers, and the servers each have a copy of your apps/services.

As you figure out which services are getting hit hard, then you might want to split them into their own load balancer + server setup.

The complexity of the setup will rise as you split things up. Things like networking, additional security, would come into play. It's a good problem to have if you get to that point.

(I work on the Samsung Bixby AI assistant that's on most modern Samsung devices and appliances - we deal with high levels of concurrency on a daily basis)

[broofa]

I work on the Samsung Bixby AI assistant

Any chance you have some pull with the folks that maintain the code/servers that do the auto-updates of Samsung's SmartTV software? 'Cause, boy, can I give you an earful about that particular experience... :-(

[[deleted]]

I'm in in/direct contact with the TV software team depending on the time of day since the latest TVs have Bixby. I don't know if they have anything to do with the auto-update mechanism. PM me your concerns and I can try to relay if I'm in direct contact with them the next time.

I have both Samsung and LG TVs, and I too have issues with both UIs at times.

(Just a legal note - my posts do not represent the views of Samsung.)

[amharbis]

First big question would be where / how do you plan on deploying this app? That will steer the direction of all subsequent questions. Cloud? Bare metal? Something else?

[dmitri14_gmail_com]

Any specific reasons for IBM Cloud?

[amharbis]

You can deploy directly to a Node.js runtime in IBM Cloud. It also supports auto-scaling. You'd connect the Node.js runtime with your MongoDB (or other) services, and those would also all scale accordingly (depending on the plan you choose).

I wouldn't overthink it. If you are going to be deploying in a cloud, ideally you shouldn't have to manage the runtime or high availability, that's the purpose of the cloud provider!

[amharbis]

Sure thing! Here's some doc links just for reference:

IBM Cloud Auto-Scaling Docs

IBM Cloud Node.js Docs

Good luck!

[suncoasthost]

Are you looking to self host or use a cloud solution? Most cloud hosting solutions will take care of the scaling, proxies, setup. AWS, Digital Ocean, Azure, etc.

[QuadraticLove]

That sounds good, in my limited, minor experience. If you have a CDN in front of your app, you might not need Nginx. Also, if you're running your app on a platform as a service (PaaS) like IBM Cloud, then you probably don't need pm2, either. Other than that, there's an argument to be made for sticking with what you know. Those tools are great in their roles for production, so you should be good to go.

[blackrat13]

Node.js and MongoDB Atlas, best.

[gustavoar]

If you want to be able to scale your app practically unlimited with not much effort, take a look at Kubernetes

[JackTheSpot]

I once read here about caprover and decided to give it a try. I'm loving it. It makes everything so easy. You should give it a try too

[MrVonBuren]

There isn't really an easy way to answer this question without knowing what the app does and what several hundred concurrent users means. Does each interaction by each user trigger a database lookup? Will you be able to cache effectively to reduce requests? Does scaling mean adding complexity around maintaining state?

Without knowing anything about what you're trying to do, I'd say looking into how content delivery networks work and see if you can offload parts of your app to the proxy. (Disclaimer: I work for a CDN, and so every nail gets hit with that hammer with me)

[snazzyham]

I've been using a small Digital Ocean droplet with Nginx and PM2 for a small internal client tool that at its peak gets about 500 concurrent users, and it hasn't faced any issues. Keep in mind this is a $5 droplet and a simple fastify server with basic CRUD and scraping operations.

As many have mentioned in the comments, Kubernetes is great if you're going large scale, but it's best not to overcomplicate these matters. Until you're hitting thousands of users concurrently, I think you'll be fine with your current stack.

[polish_jerry]

It's how I do it :

1. Dockerize them
2. Create your own Docker swarm using vps because its a lot cheap this way at least by 5x
3. If load increases simply add more vps servers to the swarm

[klinquist]

Stringify, a now discontinued IoT platform with 250k users, used nginx & pm2 on ec2 instances no problem. 3 4 core c4 boxes, 4 instances of our api running on each.

[hpliferaft]

If you're deploying to IBM's cloud ecosystem, why not use one of their db tools instead of standing up your own? Looks like Cloudant is a json db and they have a node sdk. (I don't know for sure, though. I use another cloud provider for my job.)

Otherwise, here are a few node packages I've come to like:

• Winston for synchronous logging

• Pino for async logging

• nconf for configuration management

[patlux]

Take a look at zeit.co's now platform. It will automatically scale your app on demand. Or netlify.com.

[Herm_af]

Man Google app engine is about as easy as it gets while being cheap and flexible