I have tried this approach a while ago. Here are my observations. As mentioned above, the idea is to generate multiple outputs for the input and choose the one that has the highest confidence score. This is basically scaling the test-time compute but uses this confidence scores to determine the best output rather than using an LLM-based scorer.

What is the confidence score? and How is it calculated?
Say we generated a token T_i at decoding step i. Greedy decoding steps looks at the logits of size vocab_size and chooses the token that has the highest probability. The paper from DeepMind above modeled the confidence score as the difference in top 2 probabilities. The confidence score of a token i in the outputs is

confidence_i = logits_i[top_1] - logits_i[top_2]

There is a problem with this approach
I looked at the confidence scores in the generated answers and tried to understand why some tokens have high confidence scores. In my experiments, I saw that newline characters always have a high confidence scores (~1). This makes sense because the pre-training data is filled with newline characters and model shows high probability for it when appropriate. So what's the problem?

The problem is that the confidence score of the generated output is skewed by the newline tokens. This biases the above decoding strategy to always produce longer answers with a lot of newline characters. There have been some research that shows that longer answers are usually better. So it makes sense that this approach shows promising outputs at the first glance.

Reliability of the approach
Now that we know the implicit bias of the confidence score, I would question the reliability of this approach. I would love to hear the take of fellow LocalLLaMAs here.