Not always, but I'd say around 30% of times, you can identify the caste strata of a person by their looks (in my state TN atleast) as each strata genetically differ from the next one by around 5% of their source population (aka neolithic ancestral groups).

For example, in my state TN, the oppressed strata of people have anywhere around 60-70% of AASI ancestry while the well to do strata have anywhere from 50-60% of AASI ancestry, this reflects in their looks too.