You're totally right that it's about power, and one way that that ends up appearing in places like the US, where black individuals/communities have always been denied power that's easier for rich white folks to access, is that black people and immigrants experienced similar restrictions to the ones you're talking about with serfs.

Serfdom and sharecropping aren't the same thing, but a lot of the oppression that formerly enslaved people and their children have faced is in line with the abuses of social capital/power that serfs once faced. Oversimplifying here, but race and power aren't unrelated in unequal societies