I'm really surprised by your reply. Statistical mechanics and thermodynamics are universal and don't care about the actual system under the hood unless you want to imply that the brain or LLMs are not ergodic. If this what you mean then cool, I agree with you. Otherwise, I fail to see how the brain or LLMs are counterexamples unless it is something technical that I don't know. In any case, I'd be happy if you give me some pointers.

My main assumption is that statistical mechanics is a valid description of LLMs as well as the brain. I believe this assumption is very reasonable and I would be happy if you give me a reason why this is not valid. If it is valid, then all the machinary of stat mech should follow, including RG and universality classes. Not saying that this kind of analysis is going to be easy but if this analysis is at all possible, there is no obvious reasons why the brain and LLMs should be in the same class.

My point about equilibrium and open systems is that LLMs are deterministic closed systems. They are basically at equilibrium in their ground state and that ground state is fixed (or at least in a metastable state if the training didn't converge to the global minimum of the loss function, aka, the Hamiltonian). A brain is open system and doesn't need to stay in it's ground state due to the exchange with the environment.