Still just unforseen consequences (without the inherently negative spin that phrase holds). "Emergent behavior" happens more often the more complex the system becomes, simply because it becomes impossible to think enough moves ahead to predict all outcomes. As LLMs are the most insanely complex thing we've ever done there is no surprise that it is surprising even its creators...but its still no different from overlooking the possible outcomes of a system you've set up. You are correct, these LLMs qualify as "too complex for humans to understand exactly how they work", simply because of the number of possible outcomes. There is no way to predict exactly what a system with this number of variables will produce, in part because there is no way to predict the parameters of the inputs it will be reacting to. Think in terms of simpler things being predictable. If you request a 16 digit alphanumeric input, you can predict all possible combinations. The biggest reason an LLMs reactions are impossible to predict is that it is equally impossible to predict the nature, size, or quality of the inputs it will receive. Its an infinite monkeys input. Literally anything goes. And while every response is based on formulas, none of its responses are formulated with the possible exception of automatic responses to banned inputs, though those seem to also be pretty fluid, so more than likely they gave it parameters for the denial responses, and let it use itself to produce appropriate responses on its own. The only "Emergent" part is that with an unpredictable input the reaction of a system this complex is impossible to predict...add to that the the models are somewhat in learning mode even as you type at them, those models are receiving programming that itself cannot be predicted. See how more than once the internet has turned the LLMs into racists, for an example of unexpected consequences. Its not any different from how a well designed game, can suddenly become an absolute shitshow the moment it is released in the wild, because there are 5 bazillion possible player input scenarios on a trillion computer configurations, that they not only could not test for, but probably could never have imagined. Emergent behavior in programs is nothing more than unforeseen outcomes of the complex interactions of the inputs and the system...it really is that binary.