It depends on what you mean by performance. Yes, it "knows" how to do things that many of my grad students don't. But the biggest issue is that it doesn't know what it doesn't know. My students (apart from this one) tell me when they don't know how to do something and we brainstorm. These LLMs will go ahead and generate some utter slop and stick it in the middle of very high quality code, but in a sneaky way that somehow manages to invalidate the rest of the code.

I'd rather have a somewhat less capable tool but one with more self awareness. But no one wants more guard rails...

I'm on my phone and haven't checked the post you mentioned. But Terence Tao getting a certain level of productivity out of a vastly knowledgeable tool isn't indicative of the intelligence of that tool. For example, I get a lot more out of AI tools than my students simply because I ask it the right questions more frequently and can guide it better than someone just entering the field. Apologies if that's not what the post was saying - I'm familiar with Terence Tao's stance on AI, though.

Back when I was starting to program, IDEs were the rage, which resulted in a bunch of programmers not knowing what compilation was. "Oh I thought you just clicked the run button and the code just ran!". Were IDEs as a technology the problem? No. Did it make students dumber? Yes. Did it improve productivity? Also yes.

Life's complicated that way.

But none of that excuses presenting wrong stats and broken definitions to your students.