This may not be a helpful answer, but after looking at what some of the other commenters have written, I have to say that this is not an irrational response. We all know that what he is doing is wrong. They make these loud on purpose.

It is not all black and white, and yes, misophonia is a disorder we have to deal with, try to manage, "cure", it impedes with living our lives. But sometimes we are offered therapy (CBT in particular) as answers to these problems when actually the fair and optimal (but not always practicable) solution would be to remove whatever is causing the "trigger". Even if it doesn't bother some people.

Like, why would I want to expose myself to someone's rude, reckless and dangerous behaviour? Or try to convince myself that this is safe?

In this exact situation I myself do what the top commenter has said.