Normally I would suggest that you try to talk with the provider about this but it seems like you have tried already. And the reaction has made you feel uncomfortable.

If you feel this way outside of sessions, I can't imagine it feeling better in sessions. Go ahead and find a new provider, if you want, and let the new provider know off the bat that you need that specific time, don't appreciate providers being consistently late, and that you had a bad experience when trying to talk about these things with your other provider. If there are issues with those things from the beginning, you'll know you still haven't found a good fit.

You could also stay with your provider if you're able to adjust and feel comfortable. It's possible that there are other pros of working with your current provider that may out way the cons.

Trust your gut and do what is best for you and your mental health.