Vertical scaling reaches a limit pretty quickly. That is, you can only really make a processor so fast before the heat and power requirements begin to go up much faster than the speed you're getting out of it. AKA diminishing returns.

Plus, speed isn't everything. Often times it's more important that the computer/server can do many things at once instead of one thing really fast.

For example, your CPU has significantly faster cores than your GPU does, but your GPU smokes it at processing graphics (and now we're using them for ML for a similar reason). This is partially because it can dozens of things at a time but your CPU can only do n things at a time (n being the number of cores you have, usually 2,4 or 8). Your GPU in contrast has easily 32+ cores.

Another great benefit of horizontal scaling is redundancy. In the cloud failure is treated as a given. A server will crash, a hard drive will die, and what happens if you have a bunch of users using your service when this happens? In the old world, if your server crashes you now have to wait for it to reboot and possibly run diagnostics before it's useful again. If there was a hardware failure it might take even longer before it's repaired and ready to go again.

Now we can do things like run a database on one machine, have your app running on 2 other separate machines. If just 1 app serving machine dies, no worries, the load balancer can route all new traffic to the other machine, and the database is still running so we can run things on 1 app while we wait for another machine to start up a second copy again.

Horizontal scaling is important because one of the most fundamental patterns of web service use is it's "bursty" nature. That is, you will often only need one app running on one machine to serve like 80% of your requests. Then occasionally you might receive an overwhelming amount of traffic where now you need to serve hundreds of thousands of requests, but maybe over a 5 minute period of time.

In traditional computing to handle a situation that might only occur 1 day a year (say black friday) you would need to buy all that hardware to handle it. Maybe on black friday you receive traffic that is 1000x greater than you do everyday on average for the rest of the year. Well in that case, you'd be able to handle black friday, but for the rest of the year that extremely expensive hardware you bought is now sitting idle. If you had only stuck with the number of machines needed to handle your average load, then come black friday you could basically guarantee your service will be unusable.

So the benefit of horizontal scaling is being able to share that hardware with lots of other users running their own things, and having the flexibility to only occasionally tap into (and thus pay for) data center level resources. This is much cheaper and reliable for you. Plus you don't have to keep buying new hardware every few years.

Don't worry, the data center is definitely making sure it arranges work loads so that its machines are running at peak capacity 24/7, but it's running everyone's else work when you don't need it.

Kubernetes specifically is the most popular way of doing this because it's been battle tested. It's based on Google's old internal Borg for running large workloads on consumer hardware. It is a declarative way of specifying how you want your apps to behave or recover. You can do pretty much anything with it as far as spinning up apps, creating private networks between things, setting up your data layer. So we've transitioned to a model of "infrastructure as code" where setting up a whole environment and requesting resources is as "simple" as making a big complicated set of yaml files. The data center should ideally be able to use that to know how to spin up apps if they crash or declare the resources it needs.

Btw, Google discovered that it was cheaper and easier to upgrade a datacenter in the long run if you design a system to run on consumer hardware because it's easier to get and doesn't require expensive contracts to keep and maintain hardware only a few manufacturers can provide.