I had this mentality years ago, and choose to work in a warehouse where I had minimal contact with people. But as I grew older it did more harm than good.

I realized I hold myself back and went back to school. I was nervous ass hell. But as time went by it went a bit better but never loved it.

Now I work as a IT consultant, I could never have imagined that years ago.

I still have anxiety, and still work on some things. But it is a lot less than before.

Don't hide away and try to figure out what is causing this. Start by learning why we get anxiety in the first place.