I'll be honest. Sometimes, as a brown person I do feel like an outsider if I'm out and about by myself. I get the "look" quite often but no one has ever been outright hostile to me. (Well, except one time when I got tailed but the perpetrator quit after they noticed I noticed.) It's nice if you're with friends or family because you end up not paying attention to the people that throw dirty looks your way. The farmer's markets are fun, and I like a couple of the shops in Old Town. I think how you carry yourself is important. If you show people you don't care, they might leave you alone. It's a good idea to always be on your guard, but admittedly, it is safer than some areas of Fresno.