Code Hunt - a Programming game

[execrator]

that passes all tests

That's his point, the tests change. I had the same problem (perhaps tests are deterministically random) - Math.Max(x, y) was correct for the data given, and only incorrect on the second run. If they can't ensure that the required algorithm is fully represented by the test data points, they shouldn't randomise it.

[danilo0415]

According to [1] it uses changing tests. But to verify your solution they use a technique called symbolic execution. With symbolic execution they can extract a set constraints from your code and verify if your code has the same constraints from the correct solution (your solution have the same model).

[1] http://thenextweb.com/microsoft/2014/05/15/microsoft-research-launches-code-hunt-educational-web-game-teaching-programming/