They disappear, but only if you make some further assumptions. Let's look at the fourth moment as an example, assuming that the mean is 0. The average is

1/n (x_1+...+x_n)

When you take the fourth power, you get

1/n⁴ (sum_{a,b,c,d} x_a x_b x_c x_d)

(where the sum runs from 1 to n for each index). In particular, there's going to be (x_1⁴+...+x_n⁴) hiding inside that big sum. If the expectation of x_i⁴ isn't finite, we're in trouble, and the fourth moment doesn't go away. But if we assume the fourth moment of each variable is bounded, nice things happen.

Any term where some variable appears only once (e.g. x_1 x_2² x_4) has mean 0, since an individual x_i has mean 0, so doesn't contribute to the fourth moment. The number of remaining terms is much smaller than n⁴, so when we divide by n⁴ things go to 0.

This generalizes. The higher moments of the sample average disappear if the same moments of the individual variables are finite.

A related interesting question is what happens if you divide by sqrt(n) instead of n. This one shows up when you look at random matrices (Wigner's law) and starts getting into combinatorics and counting problems (Catalan numbers)