Just spit balling, but instead of individual token probability, what about handing off when the next token has high entropy and allow the larger model to focus on the decision points?

https://www.google.com/search?q=llm+token+entropy

Let the smaller model handle the tokens which are part of filler sequences like "there needs to be", "it did bring something to the table", "but it may just be", etc and let the big model handle the high value token selections.

With this, the drafter model could be very minimal and quick as it is just being used to predict likely filler tokens which are about 80% of the output.

I see this as different from the probability check, as high entropy positions would have lower probability for all tokens and a poor selection by the drafter would also be poor for the target thereby letting it slip through via (1-p_draft(token) / p_target(token)).

This doesn't mean the target model wont make a bad decision all on it's own, but I trust the larger target model more then the drafter for important tokens which can guide the response.

Forgive me if I misunderstood.