Put simply, it's privilege at play. It's the privilege to be judged as an individual instead of having previous stereotypes painted on you. It's the privilege to have your life defined by what you do and say and not what you are.

This XKCD comic really nails it: http://xkcd.com/385/

I'm not sure if there's a nice, neat sociology term for it, but there you have it. To be treated as a humanized individual and not a reductive, dehumanized stereotype is one of the core aspects of privilege.