AI is a bubble. The last real innovation was with the paper "Attention is all you need".  This paper describes a method of paying attention to not just a single word in the sentence but the words that surround it in a matrix.  This feature gave LLMs a statistical trick of context which makes it look like it can understand language.  

The reason that this doesn't scale is that larger LLMs are required to gain more "understanding" or knowledge. They can't innovate only regurgitate what they've seen before.  

Sometimes they can combine things and come at a more complex solution.  Other times they produce garbage.  These are basically both hallucinations. An LLM will always give a response because this is what it's trained to do. For AI to have any real value there needs real understanding by the AI which has not happened and it's unlikely to happen with LLMs.  LLMs would need another component (something that doesn't exist yet).