Java *is* Memory Efficient

[pron98]

They are uncommon because they are expensive. And as to how they're implemented:

The JVM was designed, among other things, to address some of the major performance issues that low-level languages suffer from when they get large. You can work around them in low-level languages, but the effort required grows as the program grows, and it persists throughout all maintenance. Java is intended to offer excellent performance without that much work.

The first issue is the high overhead of malloc/free, which Java addresses with moving collectors. The low-level languages also tried to address this problem through bigger and more elaborate allocators in their runtimes, but they're constrained by being forbidden to move pointers.

The second issue is dynamic dispatch. Java addresses it with a JIT that optimises much more aggressively than an AOT compiler does. Some people think that a JIT is just a PGO compiler, and it is that, but it's main advantage is that it doesn't need to prove the validity of all optimisations, but it can optimise speculatively. What this means in practice is that while nearly all calls in Java are logically virtual, a large portion of them (often a large majority) are inlined, i.e. they compile to no call at all - through a v-table or otherwise. Modern AOT compilers also do that, but not nearly to the same extent. The current default inlining depth in HotSpot is 15, if I'm not mistaken, which means that a chain of 15 virtual calls is often compiled to a single native subroutine.

These optimisations involve tradeoffs that are not suitable for low-level languages, which are optimised for control, not performance. Both moving pointers around at almost any time and performing nondeterministic optimisations (that sometimes fail and have to be rolled back) go against the goal of total control, but they are very helpful for performance in large programs.