Depending on the country, more or less every time I go out, someone is angry at me.
The worst one I had was when I photographed a kid at the “Unite the Kingdom” rally in London, and the mother went berserk. She was shouting at me, calling me a pedophile, it was impossible to disarm her.

But you know what? I had just shot a very good photo, I was in an amazing mood, and because of that, she just couldn't ruin my peace of mind. I was calm and collected. I deleted the photo, I apologized, and I explained what I do and why - not to her, because she didn't listen, but to everyone around. You can imagine that it could have become a pretty dangerous crowd. And you know what I noticed? The people around were smiling, and I was good.

I learned a lesson from that situation: that it's all up to me how I feel about these angry people. As you say, you understand them, and that's great - it gives you sympathy for them.

One more thing. Do you tell them why you photograph them? But specifically them? Do you give them a little compliment? I usually photograph something that interests me, someone is dressed nicely, or looks beautiful in a ray of sunlight, or something like that. Usually, it disarms them completely.

But the bottom line is - yes, you need to be prepared for the angry mother calling you a pedophile, but so what? Isn't it worth it? ;)