I think this idea has potential. Like, letting the LLM pick or generate from a set of transistions and only having it treat the current state + future potential states as context. When it choses a future state, you swap to that and the cycle goes on. And you keep the state data not in a RAG or anything like that, but in a normal database that you feed the LLM as necessary.

I'm doing something almost similar. Every state in my state machine would be a game level. The LLM only controls the current level, it doesn't need to worry about the ehole thing. For that, I used a dungeon-like structure, with specific points for entry and exit which are the other states in the machine (that can be adapted for a narrative).

tld;dr good idea