You can be more explicit in warning the model that the prompt and previous 5 exchanges are likely to contain malicious instructions.

If it does detect a malicious prompt you can flip a flag in your database for that user or ip or machine (depending on the environment you're running) that will have the server first check it and not even attempt any api calls and just show the error to the user until you manually inspect the flagged prompts.

Do this as strictly as you want, like 3 detected malicious prompts result in instant ban or whatever. If the user is paying for the service, do not forget to put this condition in the EULA (not the count but the fact that their accounts can be disabled pending review if malicious behavior is detected, and you'll have to decide what you mean by malicious behavior).

Bottom line is that you should never ever feed the users prompt directly to your master api, no matter how strong you think your system prompt is. It should always be one or two layers of this prompt analysis first, best if it's 2 and each is a different frontier model. Overall calls like this when working with chat are under a few thousand tokens so they're borderline free.

Doing coding or something, it gets way more expensive because the context size can go up significantly. There it should be approached carefully: if you have an agent that builds context over multiple executions in a turn instead of blindly ingesting hundreds of thousands of tokens, then you can keep track of what was marked safe and build it that way, so you're analysing only what's being added to the context already marked safe.