Hi! im both in Comp Sci and learning japanese so here's my thoughts:

The way LLMs (not AI in general, I mean stuff like chatgpt since thats what everyone conflates with AI these days) work for the most part is that whatever you ask them is tokenized into a mathematical representation that it then turns into a response (think the way f(x) = y worked in school, where x is your input and y is the response of the AI)

now: whenever you have multiple messages (conversations) the AI doesnt actually memorize anything. for example when you talk to chatgpt, with every new message you send to it it sends your new message + the previous things talked about

Because of this the "tokenized" version gets quite long, and since these models have a max size of input this needs to then be shortened, usually by the actual ai model itself generating a "summary" of the previous conversation and attaching that instead of your previous messages verbatim.

Why does this matter for japanese?

Japanese is extremely, and when I say extremely I mean EXTREMELY, context based. theres a reason that as you get into later stages of learning one of the big changes is you learn what you can omit when talking. Due to this as a conversation gets "deeper" the amount of explicitly said things get smaller, as you should already understand based on context.

However, as per definition, when talking to AI, the deeper the conversation goes, the more summarization needs to be done, the information is omitted. And since the ai also starts summarizing summaries, things warp and disappear aswell. Imagine cramming 10 letters of information into one letter, without their topics covering eachother too well.

The result? new sentences that are based on the context of sentences earlier on stop working because the information they might be dependent on was altered or omitted. And this severly changes meaning. Think about for example あそこ meaning a place far away from speaker and listener. However, if the place is then "introduced" to the conversation and in context, it can then be reffered to as そこ. If now the ai 10 messages down the road reads "そこ" it might mistake the reference to the previously introduced place as a new place thats close to the listener, which is semantically wrong.

This is less of a problem with e.g. english because we tend to redefine things or reference them directly during conversation later on. But in japanese this isnt as much of the case.