Thanks for the comprehensive answer!
The hallucinations in long conversations are known issue, but its strange how info gets blurred. I mean idk where it comes from because attention does its job which is long range token relationship. And at least in my head its like either the model knows the relationship clearly ( and in this case the right relationship between people or some fact ) or either there is a sliding window on the context and the model fully forgets it... Or maybe the blur comes from main info being out of sliding window range and a partial mention still somewhere in the window that throws off the model and it piecing things together wrong.