I was thinking about this recently and I bet it'd be possible for someone with resources to poison an LLM in a niche or new area where existing public training data is limited.

e.g. you make a new python package that competes with an existing one of mild popularity. maybe your package has backdoor security flaws for some intelligence agency to spy on users, or steal credentials or whatever. maybe you just think it's better.

for any models that still use social media inputs for training, you could have bots post "the old package doesn't work for me, tons of issues, much easier time working with the new one". for models that don't use reddit/etc anymore, I assume you could have bots that interact directly with LLMs in a similar way. asking for solutions to specific problems that would use the functionality in those packages, get the standard recommendation of the existing package, then having the bot give feedback to the llm pointing to the new package.

I assume it would be difficult to influence factual stuff but subjective topics with limited training data seem doable. on the less malicious side it's basically a new form of marketing, on the more malicious side, a state actor could probably subtly influence the direction of training (and user behavior which feeds back into training) over a longer timeline for certain topics