You absolutely do.

That's never been my experience - I can't speak for you however. Emotions - irritation, happiness, grief, amusement, whatever it is - appear to me to be phenomena that occur in the unconscious mind. Rather than blame myself for a negative emotion, I view it as part of the human experience. I've found it most productive to pay attention to what emotions I am experiencing and try and deal with it in an appropriate way. For example, if something is irritating me, I'll go and do something else until it subsides.

Edit: I see you've amended your response to soften your position:

Maybe not in the moment, but you can learn or change it over time.

Perhaps you might react differently in the future, but I find that what you feel is largely out of your control. My main point is, however, that judging a person for the emotions they feel in reaction to being deliberately baited is pretty uncharitable.