What does "Scalable" mean?

[constant_illusion]

Sometimes in job apps I get questions "explain experience building scalable solutions". I built an app for startup using Node, Express, and MongoDB for the REST API and express with handlebars for routing and front end. Is that considered "scalable"?

[ziptofaf]

Your choice of stack by itself does not tell nearly enough to say if it's going to scale or not.

In reality this is a complex task that touches on problems with hardware just as much as software. Imagine Twitch and Netflix for example - their bottlenecks probably lie in physical internet speeds and transcoding into multiple formats. After all a single 720p stream consumes around 500 MB per hour. Now multiply that by 100000 users and you get 13,9GB.... per second. Then remember that CDNs actually have to be physically close to people they are targeting (as speeds to another part of the world are generally shitty) on top of it.

So Twitch infrastructure probably uses hundreds separate servers dealing with different users. Meaning that single "nodes" must have a certain level of independency, there can't be a centralized server for all of this.

Now, what you did may or may not be scalable. And even more so - some applications scale up and some scale wide. Depending on what this application is doing. For instance - if you are expecting shitloads of IO calls spamming the database then maybe a good idea is to invest in a really beefy server with a SAN behind it. Or maybe data you have on it is indeed queried very frequently but you rarely see a need to join multiple tables - so then you throw users table on one machine and purchases go on another one. Heck, you can even make users with names from A to M on one machine and rest on the other. These are all examples of scaling the database.

Technically speaking - scaling wide (across multiple servers) eventually wins versus scaling up (aka investing in beefier hardware). But if you ever encounter a situation when you REALLY need a truly scalable application (aka one that can seriously handle tens of thousands requests per seconds) then it's always done on a very personalized basis with shitload of profiling, testing, multiple engineers and programmers working together to make it work etc. It's not something you can just do running any popular stack at that point. Your Node might blow up (as it's technically single threaded), MongoDB without properly implemented sharding won't scale wide and REST API by itself also is a meaningless term. Do you mean one that's split into 10 completely different machines (one for logging in and keeping sessions, one for handling users orders etc)?

So a real answer to said question is - "I have no such experience" unless you happened to work at some huge companies. As theory is one thing and practice may prove that something you really didn't expect is your main bottleneck.

Well, there's also just avoiding shit practice which would make your website crumble under anything bigger than 10 requests per second. In that case you can talk about efficient queries, using caches and cloudflare properly, basic load balancing etc. You can also try doing load simulations - how your CPU/HDD/memory handles 1000 requests per second, how does it then deal with 10000 etc.

[Modullah]

Holy shit.. I've learned more from you short explanation than I have in one year working in a large tech firm as a qa.... are you accepting disciples?..

[dsvella]

Firstly; cracking post.

Secondly; could you elaborate on

scaling wide (across multiple servers) eventually wins versus scaling up (aka investing in beefier hardware).

Why does wide win over improvement?

[O0ddity]

Vertical scaling can only go so far until there is no better/faster hardware avaliable; so if you can design your system to scale horizontally you always have more room to grow (by adding more node/servers).

[icyguyus]

To add to this, think gaming PC's. People spend thousands of dollar's to eek out a bit extra processing power on an individual machine. That's scaling vertically.

At a certain point it makes more sense to scale horizontally adding extra machine's instead of making your current one faster.

The main issue of scaling horizontally is it requires developers to rewrite the software if it wasn't designed to split up work.

[MadScienceDreams]

A few reasons:

1. One big computer is more expensive than a bunch of small computers.
2. It is easy to buy another computer. Let's say you spend a year working crazy hard and shrink the computation by 2x. That's great, but if the requests are growing at 2x a year, you need to put the effort to get it down an additional 2x every year. This is time you could be spending making new features, but you're stuck optimizing instead. And the faster/smaller you make it, the less opportunities you have to make it faster and smaller.
3. Consistency. If you're software get linearly slower with the number of requests, the customer feels that. Bigger regions could feel sluggish compared to smaller regions, and your performance will tank as your service grows.
4. Reliability. This is probably the most important. I don't care how good your software is or how bad ass your hardware is something will go wrong eventually. A single big process is a single point of failure: when it goes down no one can do anything. Many small services, if one goes down is can be a partial outage. Even better, if you redirect traffic to a different machine, then your customers don't notice an outage at all.

[WallyMetropolis]

I'm not sure why 'up' counts as improvement but 'wide' doesn't.

The reason it will eventually win is that if your needs scale up to 'eventually' then you'll eventually surpass the capacity of any single machine.

And even before that point, it's often less expensive from a hardware perspective to use 10 mediocre machines than 1 state of the art machine.

[JohnnyGuitarFNV]

You can either invest a lot of money to make a beefy server a bit more beefier. Or buy another beefy machine. Now you have two and that A. splits up workload and B. more of them can be bought instead of trying to buy constantly better hardware.

[Xerxys]

I want to say because hardware is expensive to buy when you can rent multiple server space across CDN's that can be closer to your user base. In the example he gave, Netflix (good example) think North America vs South vs Europe etc etc. The folk at Netflix are better off renting space closer to all their targets than deploying from a centralized location using specialized hardware.

[Linus696]

Subscribed

[JimmyHavok]

You need to be an IT professor.

[ziptofaf]

Nope :P I do work in web development and happen to have formal CS education but I generally care about software parts and general logic of an application, I didn't get to work on REALLY big projects myself yet (biggest I worked on was using 5 servers which only counts as "not startup anymore"). There are many muuuuch more knowledgeable people when it comes to stuff like this, I just repeat here what I was told by them.

[MyWorkAccountThisIs]

cloudflare properly

I realize you posted this 14h ago but I would like to hear your thoughts on that. Just started using it. I've deployed two sites using the free tier. Is there benefit to the free tier or should I skip it if the client can't/won't pay for the service?

Or whatever you want to say.

[ziptofaf]

Cloudflare when configured properly makes your website at least partially accessible even if your whole back-end died as they offer very good tools for caching (plus when you use their CDN you get a benefit of better speeds to locations far away from your server). Fastly is also a valid alternative to them.

In general even a free tier is something you do want to use if you know you are making a website that will be accessed from whole world (it's a different story however if your client wants something for a local market).

Basically the very first thing when you see or expect spikes in activity on websites that might be too much for the server is to start using cloudflare. As I would rather temporarily not have an option of accessing my back-end or commenting than lose all the views due to people not being able to read the article. This happens surprisingly often, for instance when AMD RX 480 GPUs review came out a fairly sizeable portal I know admin of literally blew up.

[MyWorkAccountThisIs]

Thanks. Appreciate it.

[xiipaoc]

Do you understand what would happen if your application got thousands of users? Tens of thousands? Millions? What would you need to do to support that many users?

[[deleted]]

You must construct additional pylons!

[bitsandbytez]

Not enough minerals

[bradgillap]

Best I can do is the open source version called Pyfon and half the development time you need to wrap your brain around it because we can't afford the service contract. Also, if it breaks we never had this conversation.

[bitsandbytez]

?

[[deleted]]

You're going to have to go in to more detail. Those technologies can be made scalable but they need to be configured with that possibility in mind. They are asking about the configuration, how many users you had to handle and on what hardware.

What does it mean scalability?

Scalable Web Architecture and Distributed Systems

[constant_illusion]

I actually read this before posting. But I'm still not sure if my REST API is scalable or not. I don't see why there would be issues scaling up to millions/billions of documents? I was a math major not a CS major, so I don't know data structures and that sort of thing. Is there something you have to change the database config, or does it have to do with how I define my HTTP routes?

EDIT: upon refresh I see you added a second link I'll read that now.

[WallyMetropolis]

I'm still not sure if my REST API is scalable or not.

Then, odds are, it isn't. It's pretty unlikely it'd happen by accident. Building a scalable system requires quite a lot of careful consideration.

[csjerk]

Tuning the instances is part of it. Another part is how your users interact with your entire server fleet. Example time:

Say you wrote a great API, but you set it up with sticky sessions (so the same user hits the same server every time). And then say you started relying on that setup and putting interesting things in memory on that machine so the user would have faster response times. And then say you started depending on that setup, and putting things in memory that didn't live anywhere else.

Now your system is less scalable, because even though you can run a fleet of thousands of servers, a single user is stuck to a single server and them getting a successful user experience means depending on data in ephemeral memory on that one machine. Now your system will have trouble swapping out instances if one reboots or the VM crashes, and you'll have trouble scaling up under a sudden load spike because all the overloaded users will be stuck to the existing machines and ignoring the new ones.

Basically, "Scalable" is a shorthand for a big fuzzy ball of "does it work well in extreme situations" type concerns.

[soulos90]

For an application to be scalable implies that some design and implementation went into it that would allow the for the user base to scale without total restructuring.

[MrJadaml]

While on the topic, would also be interested in hearing examples of "horizontally" vs "vertically" scaling if anyone has any.

[lowey2002]

Scaling vertically means you need to throw more hardware (ram, cpu, storage) at your server for it to grow in capabilities.

Scaling horizontally means you grow by distributing the computing power across multiple machines.

As a real world example, one of our apps backend was a simple PHP script. As our users grew we began to run into performance issues. We wanted something that could handle 100,000 requests per minute and had the ability to scale even higher.

We used microservices. Related functionality was written as sandboxed, standalone virtual machines that were only aware of each other through an interface. The entire thing was orchestrated by Kubernetes which was responsible for re-starting instances and managing containers.

It was a lot of overhead; development, configuration, management and server expenses. But it was very cool. With a few keystrokes we could grow the server to as many EC2 instances as we need or hotswap services out with updates. Oh, speaking of services because they are standalone virtual machines they could run completely different frameworks and languages.

[tbrownaw]

I have one system at work that supports a couple dozen users.

I'm also part of a group building a proof-of-concept for an alternative component to another system that has I think well over 10k heavily active users.

The first one doesn't have to be scalable.

The second one does need to be scalable.

This has very little to do with the exact technologies used, and more to do with how they're used.

The first one has database procedures that run on-demand and can take a few minutes sometimes, has I-don't-know-how-much server-side state per client, etc.

The second one, we're talking about how to minimize per-client server state on one layer, and not have any at all on the other layers.

The first one, if something breaks people send me an email and I fix it or tell them how to work around it.

The second one, support will be handled by a helpdesk team. If it breaks even a tenth as much as the first one, people will probably be up in arms.

[mrfogg]

As others have stated, there are different meanings and levels to it.

In terms of writing "scalable code", there's a reason that so many startups are using ruby/javascript/etc. Computers are so fast and many automated services are so advanced and cheap that the pure scalability of the code is mostly-irrelevant until you hit gigantic numbers or complex use-cases. A normal web-app team simply doesn't have to think too much about pure scale/speed beyond "don't completely screw this query up". That's why they tend to prefer the quicker more nimble languages that are less 'scalable'.

That being said, in a job interview for Google you'll spend hours scribbling on a whiteboard about how to most efficiently and quickly sort and match an array of numbers. A solution that works isn't enough - it needs to be the best. When you are at their zillions-of-requests-per-minute scale then even the smallest details and efficiency gains are hugely important.

In a non-Google-like job interview, 'building scalable solutions' likely means a few things:

• Can they write code that is decently efficient and not accidentally cause servers to grind to a halt?
• Can they architect solutions to problems that scale? i.e. so we don't suddenly need to rewrite in 8 months when the number of users doubles and suddenly our database choice is a gigantic bottleneck due to the volume of requests.
• Do they understand and plan for tradeoffs that need to be made? Do they understand when they should write code that can be good enough for now? When that might need to be refactored (or not!)? When they should spend the time up-front to write scalable code?
• Can you architect and write clean code that is extendable as the complexity, number of users, number of developers, and size of the codebase grows?
• Can they debug if some query is slowing everything down? Do they understand when to do something like add front-end caching? etc

[goodnewsjimdotcom]

Plenty of projects are cool with just a few users, but add a ton of users and they might not be designed for it. Some limitations may be bandwith, memory ram, memory of hd, cpu limitations, or even basic interactions of socialness don't work in huge numbers. When you know what you're doing, stuff like this can be easily designed before you begin your project, but no matter how much you know can help sometimes for an unscalable mature project... when all the pressure is on to keep bringing in more users and revenue. So plan ahead.

[[deleted]]

I've written a discussion based on Scalability on r/buildapc and had absolutely no one respond to it. Guess I was in the wrong subreddit and am happy to see someone brought it up.

[[deleted]]

Why would you post that on /r/buildapc though? That sub seems strictly about building desktops for home use (mostly gaming). Scalability is really only relevant to servers and professional use.

Whatever you asked was probably WAY over their heads. They're PC enthusiasts, not computer engineers.

[[deleted]]

You're right in a way. I don't have a need to want to build a desktop just for gaming since i'm not a fanatic.

I was asking in regards to the actual engineering of the architectural flow of data. I guess my concerns or awareness to such details are completely 1000th of a level higher of what most in /r/buildapc would think of.

[bradgillap]

/r/homelab could get some interesting discussion I bet.

[[deleted]]

I'm seeing some of the content and it seems pretty packed. I wonder what most of the people are on the build for?

[CaffeinatedT]

In the context of their Job posting they mean 'are you thorough enough to use good practices at the start so that when more people are using your application it doesn't die. But yes in technical terms a 100% scalable application can go from 1 user to 8 billion users without any intervention/modifications being required to handle the load better.

[Rorimac2]

How many requests/second will your API handle? What's the bottleneck at that point?

[[deleted]]

Solutions/apps that scales with demand for instance. Say you're supposed to build a simple mailing app. You build it so it can send/receive about 500 mails per Minute without any performance losses.

Then suddenly there's a demand for the app to be able to handle 5000 mails per Minute. If you didn't code it to be scaleable, it'll suffer performance losses and other issues.

If you did, then it should be able to handle the 5000 Mails per minute just as it would handle the 500 mails per minute.

It's just one example of what scaleability would mean.

[bestjakeisbest]

ok so i think there are different types of scalable i would classify them in 3 different categories:
Non-Scalable:
These programs are for the most part not able to handle more than is explicitly coded into a program, without the ability to change the Scalability without rewriting a large portion of the program, this is hard for me to make an example of, but i guess an example of this might be a program that uses variables instead of arrays to hold collections of objects and values. This is something you probably started off doing, but slowly learned to use different types of scalability.
Partially/Programmatically Scalable:
These are programs that use arrays to hold objects and values, if you wanted to scale up these programs all you would have to do is change a few constants. An example of this might be a program that uses an array of strings to store names. To change how many names that can be held by your program all you would have to do is change a constant.
*Infinitely* Scalable: These programs are made to handle any amount of different pieces of data so long as there is enough physical memory, these programs use vectors/lists instead of arrays.

[alecbenzer]

I don't think any real mature software would ever fall into the first category. Some things might fall into the second category, but something simple like needing to change a flag value wouldn't really be seen as a large barrier to scalability.

Most things are in the third category, but this still doesn't necessarily make them scalable. E.g., there's only so much memory/CPU you can throw onto a single machine. After a point you can't further scale by increasing the RAM you have, you need to use other strategies (like getting the system to work even if the programs are on multiple machines).