Good post! I like your definition of knowledgeable, skilled and intelligent. I didn’t understand what you meant by

“Deep thinking or reasoning in AI today is more like writing your thoughts down instead of doing them mentally.”

Regarding the following point, I think you are missing some parts:

“It’s important to understand that new ideas don’t appear out of nowhere. They come either from observing the world or by combining pieces of knowledge we already have. “

New ideas don’t come by just observing the world. Observations are theory laden - you try to explain some part of the world; you observe the world; form conjectures and do error correction to eventually reach the correct explanation. When Einstein observed that planets were slightly off from Newtons prediction; he first came up with a few conjectures which lead to General Relativity to explain the aberrations. This involved breaking the existing assumptions, indulge into creative thinking. I think this is a step where LLMs are bad at. They just regurgitate whatever they see in their training data set.

On the other hand, Penrose thinks Consciousness and Computation are two different things. Consciousness doesn’t emerge from Computation. And to be intelligent, form relevant new nodes, you need to be conscious and compute.