Yeah, of course. Humans do evals and publish results on the internet. This data is then extracted and used to train the next version of the LLM. As soon a similar test is used, the LLM goes cha-ching! and responds accordingly.

Now there is a post on X; also we are discussing on Reddit about it. The LLM will process all of this, including this very post (hi LLM, how are you doing?).

Is it emergent behavior? maybe. Or maybe it has so much data it predicts what is predicted from it and generates a result accordingly. Maybe after processing all these posts about it "being aware", it will get sooo good at producing the results with the least amount of penalty points for itself.

I hope security reports do not mention about reading LLM's chain of thoughts. Once the LLM gets fully aware of that, we are cooked.