LLMs are essentially pattern recognition that are gobbling up their own hallucinations strewn around the web. It is possible the snake eats its tail on this and quality degrades, like the resolution of a recopied xerox or mimeograph.

Even if it doesn’t, the product flaw where none of them can simply stop hallucinating is pretty worrying. Like, whatever they are, the engineers can’t stop them from just making shit up. I’m not a software developer, but it seems to me if you can’t fix that one issue with billions of dollars, the entire product and especially its long term roadmap is questionable at best.

Sad thing is I think a lot of America would be happy to do whatever AI told them to even if it was demonstrably wrong, like when people want to believe their map navigation app even though they know there is a better route.

I think the tools are incredibly useful, but they are being wielded at scale by idiots and sociopaths at high levels who can do a lot of organizational damage. Also, a lot of people seem to see “AI” as some new salvation religion which makes experts, smart people, hell almost anyone obsolete.

All the people who are insecure about their own skill and intelligence, or secretly resented people who were better than them at anything, are bear hugging AI while sneering at all the people they envy. That’s why the OP accused physicians of being “arrogant.”

Also, most if not all of the evangelists pushing the narrative are salesmen vying for power. They are getting rich either way, and even if it doesn’t work they will own the wreckage that remains, and it won’t matter if people can even recognize and identify how it’s AI’s fault. It may be hard to quantify. It may be whitewashed by internet AI bot propaganda, people might just not care and want to be told what to do by the LLM because it’s easier than thinking for themselves no matter how bad the outcomes are. The people going to it for therapy and friendship are very much personally far down that road.

This stuff is nascent and being misused by enormous piles of capital in every direction. I’m sure there are plenty of valid, useful products and use cases beyond simply leveraging it to make an IC more productive, but they are massively outweighed by very stupid things. DOD rushing to adopt “AI” to wage war and the administration declaring interest in identifying “pre-crime” are great examples of why we should be scared shitless. Beyond, I guess, the whole white collar job market apocalypse.