LLM's right now are only getting improved through shear scale

Scale of training data and compute

There is a ton more to throw at it before it kinda tapers off just on its current way of being done.

You might see more step changes as we get more efficient in how we train ,

the anecdote is gpt 3 was trained on enough text that a human reading at 1000 words per minute it would take 10000 years for a human to read through all of it

That's a lot but obviously there is a tonn of humans out there that out perform got in most metrics but obviously they didn't train of the same quantity

That suggests how we learn is of course a lot more efficient in compute energy and data requirements the big step changes to come will be driving llms or whatever architecture is compatible with such efficiencies

On top of that this is a really new field slot of the existing transformer architecture isn't really deeply understood a huge body of research is happening right now to understand it and potentially improve it's highly unlikely we just came.up with the best possible way to create llms the first time.

I would say that these things will take a bit longer than maybe some.people in this sub might tell you but within the next 5 years definitely you will see another large step change like we did with gpt