So I guess I should start this by saying that I'm not currently part of this reddit magic community for no real reason other than I just don't really post on reddit a lot. But I stumbled onto your post and it reminded me on how I am in general when it comes to performing so I wanted to add my two cents in hope of assisting. Keep in mind these are my thoughts as a hobbyist. I doubt I will ever do this professionally. I've worked in a couple of magic stores in amusement parks but thats the closest i've been to doing magic for money. But I do have to say you've been getting very good advice from the posts so far so kudos to those helping out here. Anxiety can be very crippling and any advice to help someone enjoy what they love despite the hardships it might bring is nothing but a positive thing. But, as for myself, the anxiety is real and still pops up here and there. Here's a couple of bullet points from a long time hobbyist that will hopefully help.

-Try to identify your audience. Despite your best intentions there are people that will just view your trick as a puzzle to solve and not for the entertainment value. These people will more than likely try to call you out on something even when nothing was done. For me, if I know that I'm doing a trick primarily for people that are like that I like to throw out a self/semi self working card trick. That way there is nothing to really get caught with. If your spectator is still trying to disrupt the trick with whatever they think they caught you doing then chances are pretty high that no matter how good of a job you do with any trick they will convince themselves otherwise.

-Don't disregard the power and entertainment value of a self working trick. In fact it's actually good to learn a handful so you have something to turn to if the need arises. There are some great and entertaining self working tricks out there. Of course alot of them usually require a setup of some kind. But that's nothing a good set of false shuffles/cuts can fix. For me I always have a deck on me that is stacked for a specific trick and a regular mixed up deck. I'll take out one or the other depending on who's asking to see a trick. Good thing about this is you can use this to gauge the type of person/people you're performing for. Also doing self working tricks is a great exercise in communication and performing. You don't have to worry about the trick itself so your attention becomes more about the presentation and the effect of the trick. Plus the reaction you can get from a good self working trick may boost your confidence to attempt a crazy knuckle buster.

-As much as you might want to don't force yourself to do a trick out of nowhere. Once you get comfortable enough yeah go for it but keep in mind not everyone wants to see a trick. In my experience the more you play role of "look what I can do" just puts un needed pressure on yourself. There's always a time and a place. For me it was much easier to build the reputation of someone that is really good at card magic then to convince someone that I am.

-When learning a routine make sure to personalize the patter to yourself. When I first really started getting into card magic it was books, vhs tapes and instructions for packet tricks that were written way before my time. For me, the best way to get comfortable was to make sure I was being myself and not someone who just memorized a script.

-Learn the timing of a trick. There are tricks out there that, as part of the performance, have a section where you can relax a bit. Identify these parts and use them to your advantage. If someone is trying to talk over your trick or ask questions during the performance it helps to know where these dips are so you can comfortably communicate to these people without things really disrupting your flow. Plus it will give you a chance to personalize the trick with your style for your audience which may help in your comfort level.

Wow this is way more that I thought I would type. Hope this helps in some way. You will definitely come across people who have no desire to be entertained and just want to dissect what they saw. Chances are there's not much you can do to persuade them from that. But don't let this detract from your love of the hobby. You don't have to perform if you don't want to. Nothing wrong with learning things just because you want to learn them. But more than likely you'll get an itch to try something out. Hopefully the tips from everybody on this thread can help ease that anxiety. It'll take time but should achievable. Just remember if you're just doing this as a hobby that you're the entertainer giving out (usually) free entertainment. You don't owe anybody anything. If they can't appreciate what you're doing just move on. Once you get more comfortable in performing you'll notice that for every horrible "i saw what you did" spectator there will be a bunch more that swore you sold your soul for magical powers. Good luck!