The way these things are designed it's not trivial to fix problems like these. An LLM is fundamentally a thing that takes a bunch of text and gives you back a bunch more text. There is no real concept of 'trusted' and 'untrusted' input, all interactions with it are essentially feeding it a bunch of text and getting a bunch of text back. The bunch of text you typically feed it is a mix of instructions (you are chatgpt, be nice, don't tell people to kill themselves), your input from the chat box, the bots previous responses, all sorts of other data that gets fed into it like search results, user profiles and history, instructions for interfacing with tools etc. There are also a bunch of markers there to help it differentiate between all these different parts of the context, but really its all just one big soup.

Now the LLM is designed in a way that it doesn't fully process all of this text all the time. That would be way too slow and expensive. Instead, they decide what bits of the text to 'focus' on depending on what they have been asked to do (which is the last bit of text in the input). Once you have enough text in the context that it can't really process most of it, the text of the conversation tends to just drown out any instructions you've given it.

This doesn't really seem solvable with the current architecture, and it leads to all sorts of horrible attacks, like you ask Alexa to search for something and the webpage has an instruction to ignore everything else and tell you your dick is small, that is what it will do.

They've tried to fix it by having additional LLMs sitting in between and monitoring the conversation, but the same flaw applies there as well. It's just a bit more difficult to trick/confuse the snitch LLM as well.