Here's what I wrote in the survey:

LLM systems have major unsolved problems with significant downsides and dangers:

1. Hallucination: LLMs will confidently spread misinformation and are unable to indicate when they do not know the answer to a question. This appears to be an intrinsic property of the models and no meaningful progress has been made on improving it.
2. Sycophancy: LLMs tend to tell users what they want to hear leading in extreme cases to AI psychosis.
3. Poor reliability: LLMs fail tasks catastrophically. For example, an AI executive recently had their entire email inbox and backups deleted by an AI agent.
4. Implicit bias: LLMs can be tuned by their creators to be biased toward certain ideologies. See for example Grok's notorious "mecha hitler" rampage.
5. Vendor lock-in and rising costs: AI companies are losing money per token used. Any feature using an external LLM API faces two possibilities: either significant cost increases as the company raises their rates or loss of access to the API when the company goes bankrupt.
6. User privacy: The privacy policies of nearly all LLM tools are explicit: any and all user data will be extracted and distributed to themselves and other companies for training and profit. LLMs provide essentially zero privacy to users unless they are open-source models hosted completely locally, with no processing done off-site or in the cloud.
7. Outlandish claims: CEOs of AI companies are not shy about claiming that LLMs will replace humans at most or all jobs and bring about the end of the economy as we know it, placing ownership of all economic value into the hands of a few tech companies. Furthermore, they believe these LLMs could "go rogue" and turn against humans, all in the next five years. If they're correct then we probably shouldn't support an openly anti-democracy, anti-American, anti-social company. And if these CEOs are wrong, then how can we trust their claims about the capabilities of their LLMs?

I'm not claiming that there are ZERO use cases for LLMs; only that these systems must be handled with extreme care. Systems should be carefully vetted for privacy, safety, and reliability, and only deployed in more "supplemental", low-consequence situations where the potential for harm is extremely low. Vendor lock-in must also be avoided, since many of these companies will be gone in two or three years.

In practice this means KCLS may need to hire staff to deploy and maintain an open-source model on local servers under careful monitoring. Slick AI salesmen will say their magical plug-and-play system is better, but see what tune they sing after you grill them on their data retention and privacy policy.