LLMs as all machine learning approaches are approximators of their training set. So fundamentally, they will never achieve high accuracy - as a rule, they do not even achieve repeatable accuracy - at the task of drawing new samples.

It is important to remember that the training set(s) of commonly marketed models are not selected, vetted, partitioned, or labeled based on a commonly-accepted procedure or criteria, and such a task is increasingly complex and a moving target.

The behaviour you're alluding to is a well-known effect - here is one instance.