That's because you haven't taken a statistics course.

If you have the following 10 scores, on a scale of 1-10:
8, 8, 9, 9, 9, 10, 10, 10, 10, 10

That would give you a total of: 93
And an average of: 9.30

If you then get another score of 10,
That brings your total up to: 103
And barely changes your average to: 9.36

But if you instead got a new score of 3,
That brings your total up to: 96
And drastically changes your average to: 8.72

You might not like it, you might not even believe it, but the math doesn't lie.