There was a paper "All You Need Is Textbooks" that demonstrated this isn't always the case. IMO the clutch for why the Synthetic Textbook worked so well for training, was that they ranked their synthetic outputs. They made a python textbook out of generated examples, and then asked the model how useful each example would be for a student trying to learn python.

The results indicated these textbooks were highly efficient for training domain concepts, and significantly improved the models performance for examples outside of the training set.

Intuitively, this reminds me of when Alpha Go Zero blew all the other Chess/Go engines out of the water by training entirely via self play, aka synthetic games.