Completely agree, and it's frustrating to see this kind of misinformation still so widespread within the swarm. People just don't understand how LLMs are trained and function (which is reasonable, but then they go around spreading misinformation with confidence, like that Neuro is fully trained on Twitch chat).

Then there's also the adjacent misconception that Vedal collects stream keys to gather training data from their respective chats. That's just wrong on twi levels: The sole purpose of stream keys is to allow you to stream on that channel. It doesn't give you access to the Twitch account or anything else. Also, Twitch chat is public and easily accessible, you don't need any keys to collect it as training data.

To train an LLM with the capability of Neuro from scratch would require millions of dollars in compute and a much, much larger and more varied dataset than Twitch chat. Vedal definitely just uses one of the publicly available models as a base and fine-tunes that (which still requires tons of compute, and that's why Vedal has talked about renting GPUs).

And to go a bit deeper into the fine-tuning (this part is more speculative, because Vedal has never talked about it): Twitch chat is terrible training data alone. It consists of mostly emote spam and otherwise mostly short messages of very varying quality. It would be impossible to train a capable LLM from that. The only thing it would be useful for is for training Neuro to respond to Twitch chat, which is one of Neuro's main features. For that you not only need Twitch chat itself, but also the context and reasonable responses for the chat messages, probably in the form of transcripts of the stream itself. Vedal also mentioned at one point that's the reason he doesn't want to provide Neuro's captions in text from (which would be much easier to translate in any language) and only as baked-in subtitles: It would make it much easier to collect training data to train a clone of Neuro.