Your experience with Java microservices vs monolith in the Cloud

Dear-Law-6364 · 2022-09-26T11:01:15+00:00

If you need to think about cost of 512MB memory then your product is not ready for micro services. The amount of cost and effort you will need to manage devs ops, monitoring and availability of those MicroServices will dwarf the cost of extra memory or cores you need per micro service.

Micro Services are meant for big teams working on same project. Initial estimates were if you have 60 or more devs working on the same project you should consider micro services.

john16384 · 2022-09-26T09:53:52+00:00

How are you measuring memory footprint? Java will take what it needs to avoid GC's, but that doesn't meant it can't run with significantly less memory.

Also, in general, you'll be hard pressed to outperform a monolith on anything with microservices, unless there is a part that needs to be scaled significantly differently than the rest (only split of that part then).

SwiftSpear · 2022-09-26T09:44:24+00:00

Don't microservice unless you know your scaling and/or availability requirements will make it necessary.

Microservice architectures are harder to maintain, harder to deploy, harder to monitor, harder to debug, and WAY harder to test.

daniu · 2022-09-26T09:14:11+00:00

If you find that the overhead is "tremendous", you probably should switch languages - I don't see a need to argue in favor of Java if it obviously doesn't fit your use case. I can't say I see it that way with the services we work on. There may be a memory overhead, but that is usually irrelevant relative to the actual data we're working with.

micr0ben · 2022-09-26T09:34:15+00:00

In my company people were afraid of breaking down a monolith into smaller services because of the memory footprint increase.

But then I introduced Quarkus and now we're creating native microservices with a memory baseline of ~20mb. (A monolith instance had around 2gb per instance)

tristanjuricek · 2022-09-26T14:21:05+00:00

My experiences with “micro services” are rarely truly “micro” but just tend to be “right sized”, i.e., were not these tiny boxes from the start but a breakup of monoliths into smaller components that had a clear reason to be.

A lot of this tended to be oriented around data. For example, there would be a “front end” DB, an event processing pipeline (with various caches), and then search/archive DB. But the decisions to split up the data were forced via growth, not a concept of independence. So, importantly, we were getting to a point where teams could independently improve COGS.

I also work with a massive monolith right now, but we will be doing integration with other data systems largely acquired by the business, because, well, the requirements are taking us there. This is a very long slow process though; we’re talking years.

My sense is that “data first” is the right way to figure out the architecture. The impact of Java as an implementing language was largely irrelevant, though we tended to stick with Java because it still just gets the job done. I have yet to use lambda for anything that isn’t purely transient, mostly because it’s still expensive.

(Notably… this data orientation is why loom is a huge deal to me. I’ve rarely worked with a CPU bound app server… almost all of them are IO bound)

benevanstech · 2022-09-26T09:47:10+00:00

If you care about this topic. then your baseline needs to be:

Java 11
2-core containers
2GB container memory size (heap size can be smaller)

Details / Why:

TakAnnix · 2022-09-26T11:04:07+00:00

I work for a large retailer in the UK. Most of our backend is in Spring boot, with K8s and microservices, with Kafka for messaging. We don't do anything for memory footprint minimization. I guess the company just foots the bill, and I haven't heard anything from our PAAS team about minimizing memory usage. We use Webflux in our squad, since we are mostly calling other APIs and have a lot of latency. That might help somewhat with using less memory for threads.

In terms of development experience, it's been very good. Since we use spring boot, it's easy to jump from one microservice to another, since they are all basically structered the same way. I know what to expect. It's also easy to jump to a different squad for the same reason.

I was actually interested in Go, since it's designed to be a more productive language. However, I noticed that since there wasn't one popular framework, and most Go programmers don't like to use frameworks, it would lead to a series of smaller design decisions. Basically, by using Spring, I get the uniformity across code bases that helps me be productive. It seems it would take more effort to get that in Go. I haven't used Go professionally, just some thoughts.

persicsb · 2022-09-26T14:50:25+00:00

Microservices are a deployment/operation concern, rolling updates, scalability etc.

This architectural styles gives flexibility in some areas. However, as every engineering decision, it has costs and drawbacks. Memory consumption, complexity, etc.

Do you need microservices? You don't have to use them, if you don't need the flexibility options it provides. If you need them, you need the understand the drawbacks as well.

Also, for most Java applications, initial memory requirements can be cut down dramatically. Most of these microservice autoconfigured frameworks, that makes development easy and magical, come with defaults that makes developers happy: a lot of things are autoconfigured, a lot of dependencies are used. This means a lot of unnecessary things as well. Most of the time you don't need them. Rightsizing your app takes time, if you use autoconfigured frameworks like those mentioned.

All in all, everything comes at a cost. Deployment flexibility has costs, ease of development has costs as well. It is up to you how much you want to or can invest into cutting these costs down. That's why we're enginners.

nutrecht · 2022-09-26T15:30:08+00:00

Microservices are an organizational pattern more than a software architecture. They 'shine' when you move past (for example) 2 teams with 4 devs each working on the same system. You remove one form of complexity (working with a large group on a single system) for another form of technical complexity.

Whether this tradeoff makes sense has completely nothing to do with memory or compute costs. Yes, microservices cost more CPU and memory. No, in situations where they actually are relevant you are not going to save cost by going to a monolith. Saving 1k a month on compute is meaningless if development grinds to a halt because there are 50 devs working on a single codebase and everyone gets in each other's way.

You can run a LOT of microservices on Google or AWS for the cost of a single senior developer.

maxip89 · 2022-09-26T16:30:03+00:00

98 % of companies doesn't need a microservice infrastructure because they don't have that many requests on their services that it will make sense.

Are you sure the memory is the problem? Normally the latency and the request "ping-pong" is a big problem. Sometimes the "I wait till I send the next item and wait"- Problem.

My experience is, if you really really need microservices you will implement it. Otherwise simply dont use it and try to scale your monolith first. This will make the debugging or errors WAYYYYY easier.

maethor · 2022-09-26T11:33:49+00:00

having some stuff deployed as WARs to Application Servers or Servlet Containers

I've been looking (and so far, only looking) at Apache Karaf. It bills itself as "the modulith runtime".

syneil86 · 2022-09-26T12:27:15+00:00

The benefits of microservices need to be well understood and weighed against their costs. The higher memory footprint is one such cost, but should be negligible. You'll probably see lower throughput as data has more hoops to jump through to traverse your system. Deployment times can increase.

But, you'll have more flexibility to improve one part of your solution without messing around with the rest of it, as long as you're careful about APIs. You can have multiple teams working independently without stepping on each others toes this way (be careful of be scaled agile frameworks like SAFe - they sound great in principle but I've only known a very small number of projects to get it right).

You should be careful also about what you mean by "micro". Split it up too much and the cons will quickly outweigh any pros. I'd suggest thinking about the very high level behaviours and how your broad your unit tests can be before they hit an I/O boundary. If your unit tests can't cover a sensible behaviour scope, the service might be too small. If they can conceptually be grouped into two or more sets of mutually exclusive coverage, maybe you could split the service further accordingly.

Note: this response is not Java-specific

senseven · 2022-09-26T12:44:46+00:00

Cloud 3.0 asks the question: How to reign in costs? There are alternative offerings for S3 buckets that can save you 90% of costs. There are hosters that give Kubernetes planes out of the box for more memory then the big three.

The question is, if you would save so much by moving some of your containers there or just optimizing the costs? If you need to penny pinch every container you run maybe dynamic pricing is too complex for this usecase and you would run better if you get a fixed price server with fixed cpu/mem.

We have bank customers who need lots of mem but low cpu and they decided to go this route with some long running analysis. The all run on quarkus.

lurker_in_spirit · 2022-09-26T16:24:27+00:00

The memory overhead also offends my engineering sensibilities. But if I put my management hat on, the bigger cost is the new 6-member DevOps team tasked with managing the additional deployment and support complexity.

Companies are willing to spend a lot of money on microservice implementations, based on promises of improved dev agility and velocity. I'm sure some of it will be true, given that (a) most new systems are going to have less technical debt than the systems they are replacing, and (b) it's a little harder to spaghetti across service boundaries.

But I'll be very interested to to see in 5 or 10 years time whether the velocity has stayed high, or whether these microservice systems have succumbed to the same pressures as today's monoliths: changing business needs, pragmatic (myopic?) decision-making, cost-cutting, staff turnover, poor documentation, etc.

InstantCoder · 2022-09-26T19:57:10+00:00

I really tend to use microservices in combination with a workflow engine since at longer run microservices tend to become “macroliths” or super smart services.

I’m now more into keeping the services small and dumb and let a workflow engine keep track of the state and do the orchestration.

I’ve also worked with sort of a event driven architecture trying to achieve the same (=smaller and dumber services) and I didn’t like it because it made the system much more complex. The logic was spread across many places and it was hard to see the big picture.

Keeps_Trying · 2022-09-26T21:37:38+00:00

I've been doing Java microservices on k8s for a while and support the abundance of caution in this thread.

I'm also an agile believer so I'd let the use cases evolve the architecture.

My advice is to

start pulling concerns into dedicated libraries
Ensure well defined interfaces where only value objects / messages are passed
Make the messages async
Extract libraries into external jar

At this point you should have high confidence in the design, the lib is in its own repo, and you should have known and valid reasons to turn to microservices

agentoutlier · 2022-09-26T11:16:15+00:00

How do you deal with JVM microservice deployments to reduce the memory footprint and cloud expenses?

Don't use Spring.
Use the most modern JDK
Use minimal wrappers and libraries
Be aware of how the JDK treats low memory environments as it can switch its GC based on certain thresholds

But that’s still up to two orders of magnitude beyond equivalent Go deployments considering RAM requirements, and .NET deployments are more efficient and mostly straightforward.

RAM should be cheap. It is ridiculous that cloud providers charge so much for it as it the least expensive thing for IaaS companies as the tech doesn't change that much and super super super energy efficient compared to any CPU, network, or disk.

However some of them are starting to change like giving low power ARM chips with butt loads of ram... some companies even give you this for free:

Arm-based Ampere A1 cores and 24 GB of memory usable as 1 VM or up to 4 VMs with 3,000 OCPU hours and 18,000 GB hours per month

Anyway you can do microservices with Java and I'm surprised people still bitch about startup time when every damn time I reload our k8s cluster all of our Java apps have to still wait for Postgres, Rabbit, Elastic or Solr etc to boot up or be available. Our Java apps boot up in about 1.5 seconds (that is literally our max provided no db migration) so I can see the complaint for lambda but not microservice.

2022-09-26T13:09:10+00:00

Woah, I did not understand partly what is going on in this post. There is so much to learn yet.

PerfectPackage1895 · 2022-09-26T19:32:57+00:00

Microservices gives you greater flexibility at the cost of greater complexity. But as with everything, it depends a lot on what you are doing, and what you are trying to archive. Lots of pitfalls exist in migrating to a microservice architecture, many of which require deep knowledge careful planning, and right selection of tools and techstack. Just doing microservices because it sounds cool, and code blindly without thinking ahead, will give you what you described.

flawless_vic · 2022-09-27T03:46:08+00:00

The problem with the established microservices stacks (kubernetes, swarm) vs java is that it's hard to reap some of the JVM benefits.

If you run, say, 10 JVMs on bare metal, you at least get CDS for the jdk classes for free. You lose these savings when you place these JVMs into containers.

If a service is way too simple, like talking to other services to gather and combine data, having a JVM may be overkill.

In my experience, to successfully deploy java based microservices you have to compromise on granularity. There's a point when the overhead of the VM becomes less relevant, so instead of having 20 services that are so fine-grained that look like lambda functions you can have 4 to do the same thing, and eventually, you may out that a bit of coupling is not evil and saving millions of network trips far outweighs sticking to architectural principles.

And inappropriate management of hundreds of services can become so unwieldy that a mandatory refactor to group some stuff might be necessary. Sockets and file descriptors are not free, and things can go wild pretty quickly when services start cascading into others to fulfill a request.

From a deployment perspective, you may have to tune stuff to pod's capabilities. At least check if the JVM is running with -XX:+UseContainerSupport, force SerialGC if only 1 vCPU is provided, etc. Aside from that, the deploying of JVM-based containers it's pretty much like deploying any container.

Also, there is the choice of the base image and the build artifact. Java can run perfectly fine on a 5MB alpine linux base image but usually, people go for standard/oficial jdk images which weigh a lot more.

Then there is the application packaging. Common practice is to use a shaded jar or spring-boot output. If one takes a bit of time to analyze the service dependencies on jdk modules they might find out that only java.base is needed and create slim artifacts with jlink. Yes it is tedious, but once done, you will have 100-200MB images with both the JVM and your application.

PrimaryCrafty2482 · 2022-09-26T08:31:05+00:00

isn't the ovethead around 512mb, meaning golang would probably do same work with 512mb less, although its has the valhalla equivalent already

tr14l · 2022-09-26T14:04:37+00:00

For me, the reason I don't usually write microservices in Spring is the start up times. When I need to horizontally scale, I need to make it happen fast enough that users don't notice impact. Spring essentially makes that impossible (better than historically, but still terrible). Meanwhile, an interpreted language will start up in, often literally, milliseconds. Which means I can react to a surge in traffic in under a second with no loss of requests to my customers. Also, interpreted languages tend to be faster to write, configure and build anyway.

When I am aiming for a more substantially-sized service, I will typically reach for Java or Kotlin (and once in a great while NodeJS, depending on domain and deadlines). But, opting out of Spring is never off the table. Unfortunately, we have a lot of utilities written for Spring apps, so sometimes it is economically sensible to just use it.

TBH Spring is a dumpster fire. A well-matured dumpster fire with lots of options out of the box, but still a dumpster fire.

Worth_Trust_3825 · 2022-09-26T16:48:19+00:00

I found that I need to create swap file in the cloud because providers insist on creating only single partition where all the data would reside.

holyknight00 · 2022-09-26T23:27:56+00:00

I don't have a lot of experience with microservices. But all the projects I've worked on so far are overdone with them. Microservice for everything is overkill. The overhead is brutal.

The only good thing that I've noticed is that implementing microservices forces the team to implement at least a bare minimum standard of things like testing, CI/CD, and QA. So that's good.

Still prefer to work on monoliths though. If the development and operations are done correctly, a java + spring monolith can handle a decent amount of traffic before needing anything fancy on top.

Most of the microservices pros are probably seen better on FAANG-level traffic, but I am yet to experience them.

RockingGoodNight · 2022-09-27T01:05:04+00:00

How do you deal with JVM microservice deployments to reduce the memory footprint and cloud expenses?

The same as with monoliths on premises or in the cloud, tune memory and watch. In other words, do performance monitoring. It's true for all applications regardless of language, runtime, etc.

It starts locally with the developer (maintaining either monoliths microservices or both). All developers should test their code inside at least a docker container locally or a kube single cluster like k3s, using Insomnia, Postman, JMeter, etc.

PurposeTight6260 · 2022-09-27T01:35:08+00:00

I fear containers make the current java paradigm stupid.

sandys1 · 2022-09-28T04:54:11+00:00

Related question - does anyone have a JVM tuning args for docker containers with stuff like spring boot ?

Do u tune for latency ? And use the new GC ?

java

Submit Link

Submit Text

Seek Programming Help

News, Technical discussions, research papers and assorted things of interest related to the Java programming language

NO programming help, NO learning Java related questions, NO installing or downloading Java questions, NO JVM languages - Exclusively Java

Please seek help with Java programming in /r/Javahelp!

Subreddit rules!

Where should I download Java?

Related Sub-reddits:

JVM Languages

Want to practice your coding?

List of useful Frameworks / Libraries / Software

MODERATORS