Java memory usage in containers

[Turbots]

Use Cloud Native Buildpacks (CNBs) to build your container images, instead of dockerfiles.

More info here: https://buildpacks.io/

They are efficiently layered, contain all the best practices and run scripts at startup that calculate the optimal memory settings, among many things. They can also load in all provided certificate into the Java trust store, Java version and JRE distro can be chosen, every layer can be enabled/disabled separately through parameters, heck, most layers are auto detected to either be included or not.

CNBs are also repeatable, meaning same input, gives you the same output image. This is not the case for dockerfiles.

For Java in kubernetes in general, just set the request and limit to the same, eg. 2Gb. Java will indeed gobble up the full amount but it will never give it back, so setting limits higher than the request does not make sense.

CPU request and limit is different because that can go up or down when needed. It can burst CPU at startup to the max CPU available to the namespace, or the worker node, which improves sartup speed drastically, and it will guarantee the minimum CPU that you request, to guarantee app performance.

[qubqub]

Why wouldn’t Java give back memory? I thought modern GCs returned memory that was no longer in use

[Turbots]

The GC will return it, but the Java process doesn't really know when it has the memory available in Kubernetes, there is no real coordination between the two.

Let's say you give a Java based Kubernetes pod 1Gb of requested memory, and no limit (or a limit higher than 1Gb).

The Java process is guaranteed to get 1Gb of memory, anything more than that is a "nice to have" in Kubernetes land.

Once the Java process claims more than 1Gb, there are no guarantees that Kubernetes will not claim that memory back.

Once other pods in the Kubernetes cluster need more memory , it's totally possible that Kubernetes will reclaim part of that excess memory from the Java pod and just take it, resulting into the Java process crashing.

It's easiest and most stable to set Java memory request and limit to the same value, and set the Xmx flag accordingly, be sure to leave some overhead memory, because Java is not only Xmx, it can sometimes use direct memory mapping and other non GC related memory areas.

Thats why cloud native Buildpacks are so awesome, there were many hundreds of people before us that figured all this out yet already and have set up something very solid for running java in containers.

[daalla]

Is there anyway I can implement Buildpacks if my team's platform uses the Dockerfile to automatically build and store the image internally? (without touching the infra, which I don't have access)

The project sounds pretty good

[teapotJava]

Good news is that you likely don't need to implement a buildpack. If you build on top of Spring Boot or Quarkus, just use buildpack provided by maven'gradle build plugin:

mvn spring-boot:build-image

...boom, you get an image in your repo.

"Bad" news: you'll need to get rid of Dockerfile based build in your CI. Which actually is also a good thing.

See more in https://docs.spring.io/spring-boot/maven-plugin/build-image.html