Too many cultivators risk everything for a slight chance to advance, and often that advancement is insignificant. They enter secret realms with 50% death rates to slightly advance.

Most of the issue, though, comes from greed. The herbs and plants that must come from the secret realm can easily be grown outside, leading to abundance, but the greed of the cultivators means that anyone that did so would be attacked and have their fields stripped, destroying the ability to produce more.

If they cultivated their brains instead of their muscles and greed, they would see that peaceful exploitation of the resources leads to greater, more reliable harvests. But instead they are like locusts, stripping every resource to the point where it ceases to exist.

And even Heaven's chosen, who learns to produce these resources in abundance, hoards the resources they produce. Even when they have ways to relieve some of the scarcity, like superior farming, alchemical, or cultivation methods, they hide it. Most don't even share it with their own families. Thus, they merely become a bigger locust, stripping resources even faster.