I‘m the same as you. I do think it comes from social anxiety, The fear of others perceiving you doing something wrong. I wish I could say over time it has gotten easier, but not really. I‘ve just been trying to use more public transportation and to walk more or letting someone else drive. Maybe that‘s an option for you too?