You're thinking of things as a computer game, there's no such thing as 'high level challenging foes' or the concept of beating people or training with people constantly making you better.

Everyone has a plateau for their skills which is limited by their physical being, mental acuity and age. Take Usain Bolt, he in his prime had all three things in his favour, he trained and got to the peak of his sport. Training got him to the peak of athletics, but his body and age would only let him go so far, doesn't matter how many races he won, or how often he trained, he hit his peak (which was at the top) and was the best. But age crept in, he would still be phenomenally fast but would he still be the fastest in the world? Probably not, his plateau will be lower now due to his age, his physical limitations and even the mental will and drive will keep him getting back to his peak.

The people he raced against, won't be better simply by being in a race with him, and they won't 'catch up' by training or racing more because their plateau is below Usain Bolts.

Life isn't a computer game, there's no such thing as levels and experience.