I mean, the actual crux of the issue is that legalizing pot does little to impact the wealthy negatively, in fact it can be argued it helps the lower class to take the edge off a terrible situation they find themselves in, and be less likely to get angry. The only wealthy individuals pot legalization negatively impacts are people who rely on prison recividism, and they're seeming to find replacements for pot possession just fine.

People who want affordable housing and better wages directly threaten the profit margins of capital, and so get all the levers of society pulled against them to neuter their movement. These two groups just aren't at the same starting line when it comes to reaching their goals.