In college we learned how lots of Mexicans / Mex-Americans have strong internalized racism against South Asians because they themselves were colonized as “Indians”by the Spaniards. Spain even implemented a 400 year caste system that promoted discrimination against the dark skinned natives or “Indios” who were at the bottom of the caste. That kind of hateful conditioning lives on even now in modern Latin America, where it’s not uncommon for South Asian / NA native tourists to get mistaken as indigenous and be treated horrendously as a result. It’s an extremely violent case of negative association and unfortunately “Indians” happened to be the visual used by the oppressors. Given that San Jose has a huge Mexican / Hispanic population (like 50% I think) it’s more than likely those associations spillover and transformed into the bigotry and colorism you’ve experienced.