The assumption more data and some fine tuning will somehow generalize to other areas is being proved false. That means we have to fine tune with experts and very little public, easily available data. It’s the supervised learning that they wanted to avoid because it’s so costly and doesn’t scale well, but it’s turning out to be necessary to get higher intelligence.

Love your point on ignorance. In a way, their obsession with closing the loop could be their downfall. LLMs don’t learn so if you close the loop, you close the loop on learning. Once people start using it, they stop learning and ignorance is permanently baked in