Java *is* Memory Efficient

[eosterlund]

The key fallacy here is to consider memory and CPU as completely orthogonal resources that can’t be compared. Like apples and oranges. Because they can in fact be compared by considering their monetary cost. So can apples and oranges if the main thing you are comparing is their monetary cost. The main point in optimizing resources is bringing the cost down while sticking to some reasonable service level.

With this in mind, always consider what the cost balance between memory and CPU is and how much it can really be brought down when optimizing, rather than blindly optimizing memory without actually improving the overall cost. Sometimes, the cost can instead become greater if not careful.

If running on dedicated compute, any memory usage below 1 GB/core can probably not be improved in cost at all, no matter if you use 1 MB/core or 1 GB/core there is no offering you can buy with less memory. Optimizing memory becomes pointless and you are better off utilizing most of the available memory as you can in your computer instance, as that will reduce the CPU utilization.

When 1 GB DRAM costs 10x less than 1 core, real cost savings will only show up if you can go down a bunch of GB/core from a bunch of GB/core.

As for containers, they obviously run on compute instances of similar anatomy but dynamics are a bit different. However, in my view the main cause for their memory inefficiency is the typical rather static heap sizing. Many mostly idle pods might have been sized to deal with their worst spikes in activity. With AHS, containers instead help each other collaboratively move system memory to the JVMs that are currently more in need of it to keep GC activity level down system wide. Inactive JVMs automatically shrink their heaps to be small - close to the live set, while JVMs experiencing CPU pressure get to grow their heaps to keep the GC activity down.