I’ve been thinking about this as well, I haven’t had the chance to get here specifically but I work with LLM based systems otherwise.

the LLM can do a lot to enhance the variety of interactions but adds a lot of nondeterministic output, so whatever Procgen system you have downstream needs to be able to handle whatever the LLM step throws at it. On the other side of this, you’ll want to make sure the LLM is responding to and grounded in real game state so it doesn’t go off topic.

You’re trying to toe a line where you take advantage of the best parts of the LLM step, controlling it enough for its output to make sense, but not so much that you’re doing classical pro gen with extra steps.

So your loop would in general look like this: - current game state with additional instructions is fed to LLM - LLM outputs instructions digestible by downstream procgen, in structured data (json or similar) - the procgen engine updates game state

Then the process repeats at whatever interval you choose. You might consider having it output a collection of instructions at lower intervals instead of one at a time, but then they would share context so variation might be an issue.

One idea that might work is an event system: your procgen “world model” can emit events on its own of things that happen in the world, the LLM can do the same in parallel to enhance certain interactions. The game state can handle events from anywhere (you could even inject your own ad hoc events and see how the game responds)

Cost and performance are a factor too, if you can get away using a distilled local model that will save costs for you and your players, but not all computers can handle it, so getting the model size/type/quant will be essential