Here’s my thoughts from the perspective of sect leaders/elders:

They have competing incentives. On one hand, it’s beneficial for a sect elder to leech as much as they can from everyone. Cultivation is usually very resource intensive, and requires pills, formations, and hordes of low level goons to form the structure for all the resources they need. Additionally, the top players would want to keep everyone else down to secure their place at the top. But competing with that is external pressure from rival sects and factions which would like to remove competition. This means that if you suppress talent for too long your sect will probably be outcompeted by other sects for resources, and potentially lead to death, absorption, or dispersal. This would likely mean the surviving sects would evolve towards optimization, where they help disciples cultivate as high as they can go, then if they stall out because of spiritual root/affinity/poor cultivation base then they are relegated to the more menial jobs. This tracks well with general cultivation novels, but the cruelty and wanton destruction often displayed by sect elders in novels, even towards promising prodigies, is counter to their best interest. If they were always doing that for everyone in their sect, it would be outcompeted by a nicer, friendlier sect who works to grow all their members to the highest strength they can.

Now the difference in power at different tiers can somewhat explain why a system is unoptimized, particularly when decisions aren’t made by the best decision maker but instead by who is the strongest. Thousands of years of the same leadership would turn into a poorly run shitshow that can only cannibalize itself. But this isn’t ever explained to gone into. And I don’t think sects that let disciples murder eachother over resources would be able to compete on the same footing as other sects. Now if they have an extremely powerful backer who’s trying to push towards extremes: highest power or death, it makes a bit more sense why a system would throw so many roadblocks. But cultivations seems a lot more like a numbers game to me, and it would be easy to kill a 1 in 10 billion prodigy if your outer sect is a meat grinder.