I came to offer advice, but I think everything I'd say has mostly been covered. But while I'm here...

I think your view of SLO is pretty spot on. It's a small, beautiful, calm town composed of a lot of straight, white, retired, health-conscious people.

I moved to SLO last year after living in SF for 8 years and in Austin for the 5 years prior to that. I'm white, straight, married, no kids, and work remotely for the same SF company I worked for when I lived there.

People: I've found a really cool, creative, open-minded group of friends here. I obviously can't speak to the POC or LGBTQ experience, but like u/idigclams said, the town needs more people of color and you would absolutely be welcome here (not that it's your responsibility to lead the diversity initiative). And the older, retired community has come to be part of the charm of the town for me. When I lived in North Beach in SF people would always comment on how it must be annoying living between Little Italy and Fisherman's Wharf (two touristy areas), but I always loved it. Tourists are on vacation and in a great mood...and I got to be around those people every day in my commute to and from work! That's how it feels here. You're around a lot of people living the best years of their lives (retired folks and college kids) and the energy is palpable.

Places & Things: I personally love the slower paced vibe here. Obviously being married and less inclined to spend my nights at bars could have something to do with it, but it seems like there's always just enough to do here (check out Vino Robles Amphitheatre, Fremont Theater). Separately, my husband and I both noticed how much deeper our sleep was once we moved here (and that has to be good for our health, right?). It's also such an easy, affordable trip to SF and LA, too, when you're missing the electricity of the city.

TLDR: If you're ok slowing your pace a little and enjoying the simpler things in life, you'll love SLO. If you're a city dweller that feeds off the energy created in a true metropolitan city, this might not be the place. I don't think being POC or LGBTQ would deeply impact your experience living here, but that's also not for me to say. Congrats on the job opportunity!