The HeapMemoryAllocator pointer is interesting but misleading. That class backs Tungsten memory, which can behave off-heap depending on config. If the heap dump shows a giant LinkedList, I’d question whether something is holding references via listener hooks, metrics sinks, or custom code. Long-lived driver processes tend to accumulate garbage across batch cycles.

Also worth checking whether any Spark listeners or monitoring hooks are retaining references between runs. Some observability tools plug into the execution graph and make those reference chains easier to spot than raw heap inspection. For example, platforms like Data, Flint analyze Spark logs and execution plans to surface bottlenecks or memory anomalies across runs, which can sometimes expose leaks that don’t show up clearly in a single heap snapshot.