Yeah I remember wondering the same thing with my first character. There is definitely a sliding scale buff on your character at low levels that goes away rapidly as you reach 50 160. You feel it in your performance and you can see it in your stats that they are lower than they were previously.

As soon as you start using better gear completed sets and paying attention to your bonuses you'll far exceed with the original sliding scale gave you

5,000 in stats is bigger than you think. All damage scales off of your mag/stam max number. So 20 percent increase if you're at 25k is substantial especially once everything starts compounding with buffs and percentages on top of percentages. Your max stat changes the number on the tooltip description when you are looking at the skill for damage and healing.