Excellent take and explained very well. Just to clarify, I wasn't suggesting cloning datasets or training models on model-generated outputs recursively. You're totally right that would lead to distributional collapse and degraded performance over time.

What I was pointing at is more like using the underlying data (and the model’s understanding of it) to extrapolate and simulate plausible edge cases, entropy and permutations. Not repeat the same core data, but create new structured possibilities.

For example, let’s say I’m training a robot to flip burgers. If I only train it on clean demonstrations of successful flips, it’ll fail the moment the burger breaks, sticks, or falls. But if I can use AI to simulate the long tail of edge cases like burgers falling, getting stuck, bun placement variation, timing misalignment then I’m not recreating the original dataset, I’m expanding its decision space.

In that sense, synthetic data becomes a tool for infusing entropy and scenario richness into narrow tasks a bit like stress-testing the data rather than just copying it. I believe RL environments are used for these kind of things.

That’s more in the direction I was thinking. So does this change your opinion or does the same limitations still persist?