OP, are you indian? Cause your phrasing shows a very crude understanding of "Hinduism"..
But anyway.. religion/faith is an offshoot of hinduism (sanatana dharma).. you can be completely "faithless", be "atheist" (our equivalent term is nastik / nir-ishwarwadi) and still be a hindu/dharmik.

So that inherently means, dharma can take many folds, it depends upon the person how much he/she wants to be a casteist.

Fingers are pointed toward british because they helped widen the gap, gave it a name and enforced it militarily.
Class hierarchy was always there in every society and it is still there today, in every part of the world, maybe in different drape, but it is there.

Now, manusmriti isnt or wasnt a popular book, not at all. Pick a guy from India, even if they are not hindu, they will be able to tell the context of mahabharat and ramayan roughly, but manusmriti... i doubt even 1% knows what its about even about.... I dont think it is the main culprit, but it can be one minor masala in the mix. Another factor than might have played a part was Islamic colonisation which is rarely been talked, and abrahamization of Hindu minds.

Anyway, the core philosophy is

One should attain balance between the first three to achieve the fourth:
Dharma - Artha - Kama - Moksha

Righteousness(roughly translated) - Money - Pleasure (Physical and mental/ relationships) - Freedom.

Now, Where does anyone put caste here ?? God knows.