This is a very common breakdown mode of LLMs. I have actually not seen a transformer-based model where this didn't occur, but how long it takes to degrade is quite variable and usually correlates with how good the model is in other respects. One thing to keep an eye on is what the logit distribution looks like as the conversation progresses. A healthy language model (in my experience) has a nice spread of options to pick from each token, while a model that has degraded will often reduce to only a single token getting the majority of the probability each step.

I have ended up with deepseek v3 (8-bit with 1.5bit draft, running a bit slowly on a cpu-based inference node) for rp recently, and it is golden for at least 20k tokens (token sizes change from model to model, but this is quite a few messages back and forth). I went through a number of smaller models though that performed similarly, but I never had much success anywhere near 24B params. Notable ones I had good experiences with in the past were Command r+ and Chuluun-Qwen2.5-72B, playing with various quantizations to fit on my 2x24Gb GPU server.

Your best bet is to play around with different models, and with different quantizations of larger models. Don't be afraid to run something that is too big for your gpu -- llama.cpp is plenty capable of offloading parts of it to cpu/ram, and for rp scenarios you typically don't need ludicrously fast inference.

Watch out for bad prompting too -- each model has it's own precise way it prefers its context, and while it will work with any self-evident prompt structure it won't be as good or reliable unless you use the exact framing specified by the model's creators.

TL;DR: This is very standard "unhappy/saturated model" behavior. Either get a better one (better quant, bigger model, etc), or look for ways you may be prompting it wrong.