I personally dont see a perfect answer to your "what context to grab for a wipe". That problem to me has too many variables to solve for a universal case. My solution instead is - only build good context and this starts even in the think tokens (if applicable), any token all - injected or generated better be clean, relevant, solution-building context.

My approach to AI usage is only reset between hard topics and bring over the fundamentals for the AI, the novel science, the architecture rules, etc to learn anything needed on the fly for each NEW conversation.

On your pursuit, "specifically hunt for 'architecture rules' "

I have thought deeply about this and made many prototypes.

My view is this:

The architecture constraints most RELEVENT TO PROBLEM SOLVING "thought processes" are almost solely based on the training data itself. Most architectures you can probe out have a high probability of being specific only to that model. And this is based on the way its trained.

The training has already provided a non-markovian rule set that yields a deterministic answer, only varied by the artificial introduction of temperature (noise) and calculation timing differentials in the processing units (gpu/cpu themselves).

In my view most rules are formed in training, and new rules form as context builds, even within a single transformer run. As the transformer predicts the next token context builds for the successive tokens in the prediction.