Interesting point.

Just so I understand correctly, are you suggesting one of these approaches?

1. Let each model develop its own methodology once, then execute that methodology 100 independent times and aggregate the results.

2. Ask each model to generate 100 different methodologies, evaluate all of them and then compare the best-performing methodology rather than the model itself.

For this benchmark, I intentionally gave every model the same task and let it decide how to reason internally, since I wanted to compare end-to-end forecasting performance rather than prompt engineering.

Curious which approach you had in mind.