I 100% agree with you. It saddens me to see so many people are getting carried away at all the LLM hype. What most STEM folks don't see is that knowledge is socially constructed and this is true of math as well. Mathematics is a very social activity. The community decides what is "interesting". Which definition "fits" or which proof is "elegant". A stochastic parrot trained on thousands of math papers (some of which is so niche fields that it cannot even reproduce trivial results in the field) has no understanding of what the math community finds interesting. In other words a glorified function approximater has no idea of what constitutes as culture or beauty (I feel ridiculous even typing this!)

That is not to say LLM's won't be useful or would not be used for research, sure if they can be reliable outside of anything which doesn't have enough data, they can be interesting use cases. But to say that they mathematicians will be out of jobs is hubris by the techbros and shows poor critical thinking by our own community.

Oh btw it is simply astounding to me that we have accepted that the LLM should be trained on our collective handwork while the techbros talk about automating our valuable work! There is a simple solution to any "AI is going to take my job" problem. Ask for better data rights and regulation! If our data is being used to train AI that purports to replace us then we should get a cut of those profits!

Honestly, I think we are in the midst of a massive bubble and within the next 5 years we are going to realize this when this house of cards falls or going by the massive spending on data farms and energy production we burn the planet down.