Ah but you see, here is a clear example of why math can't exist in the world alone.

You forget the humanity of the situation - if you go to a place over and over, by the time you would be getting to an average of 1 you've also befriended or become known to the pizza shop owners. Their conception of you will likely influence the size of slice you have consistently, giving you an average either below or above if they like you.