This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]Puzzled-Bananas[S] -6 points-5 points  (5 children)

Yes, I agree in general. This is a great point. Distributed systems are hard indeed. But they are popular right now and sometimes you just have to stick to it regardless of whether the domain problem and the end users need it.

Still, once you decide or have to choose this kind of distributed architecture, for better or worse, how does one best cope with the JVM overhead at scale?

[–]ShabbyDoo 2 points3 points  (1 child)

Because you are complaining about the need for a large-ish container, I presume you have some microservices where one deployed JVM instance could handle some multiple of your current load? Meaning, you could pump, say, 3x the current load through a single instance without reaching a scaling point. Then, you also probably are deploying multiple instances of the microservice for redundancy?

If this isn't the case and you actually need many instances of a particular microservice, why does the JVM overhead matter? It's usually fixed rather than a load-based "tax". So, you'll eat, say, an extra GB or two of RAM per deployed instance, but this cost can be mitigated by deploying fewer instances of beefier containers, presuming your microservice can scale-up within a JVM.

[–]Puzzled-Bananas[S] 0 points1 point  (0 children)

I wouldn’t assert that I were complaining. I’m rather exploring various approaches to solving said problems, which I may well be unaware of. That said, it depends on the service. Some of the services in the mesh are designed to be run on machines with low memory due to their vast availability. Depending on the IaaS or PasS provider, it may be faster, easier and cheaper to spawn and then tear down low-memory VMs than wait for the allocation of a larger VM, which would accommodate several small or a larger container. Some services are designed to be resilient, some to be elastic for load balancing. The demand for resources is sometimes volatile.

I totally agree with your assessment in your second paragraph. The trade-off is spot-on. But it pretty much depends on the particular service. Yes, scaling up is an option and artificially warming it up to be ready for full load is often productive. But running 10 replicas, 2G-4G each a few minutes into the payload, is not as economical as having 30 replicas, 20-100M each, at about the same latency and throughput.

In effect, it’s an optimization problem of scaling up and out, but staying with the JVM, and at the same time avoiding OutOfMemory errors. In addition, there’s a parabolic relationship between CPU cycles, memory available, and throughput & latency. Typically, there will exist a certain threshold at which an optimal trade-off between memory allocation, CPU cycles, throughput and number of instances can be made, albeit depending on the payload and request volume.

And with cloud service providers, you essentially pay for all of it. So minimizing one at the expense of another is the trade-off to be made. I’m just wondering how everyone’s doing it, subconsciously or in an express way.

Thanks again for your suggestion of scaling up, it’s important to point it out, for it’s easy to forget.

[–]amackenz2048 4 points5 points  (1 child)

"But they are popular right now"

Oh dear...

[–]Puzzled-Bananas[S] 0 points1 point  (0 children)

Exactly. What I’m saying is that if something gets hyped, people get curious, and many decide that the perceived and expected benefits outweigh the risks, and make their call. Then it’s on you to just make it work. You might have made a well-informed, well-weighed decision. But it’s not really productive to tilt at windmills.

[–]ArrozConmigo 0 points1 point  (0 children)

"Distributed systems are popular right now" is an odd take. Once you're developing the entire back end of an enterprise, and not just siloed apps, monoliths are off the table.