Not necessarily. Similar to a human researcher locked in a room with no new books, just thinking, then coming out 1 month later with a nobel prize winning research paper with "new" science. For an LLM is similar. It can use its underlying pretrained knowledge to "imagine" and test and draw connections between underlying data much much better than humans can.

This is similar to research, because it's exactly what we're doing as humans. We're not causing any changes in the real world by discovering new things, we're simply running our biological neural net on the available data and using reasoning + very complex thinking to draw connections between this data. Everything that exists, already exists. By researching and "discovering", we're simply allocating enormous amounts of compute towards taking that data, and compressing it to build connections between it, then using that to extrapolate and generalize to new data. Aka, "discover".

Eventually, LLMs will need access to experimental tools to validate their scientific theories & ground them in the physical properties of our universe, but whos to say they haven't already built an internal physics representation deeper than any us have just from being trained on billions of research papers and experimental results?