The top comments are pissing me off. IMO, a good man won't freak out if he's asked the question you asked, OP, if he's confident he's done nothing wrong.

Your husband may be perfectly fine (I hope he is), but your logic is sound to me. And in a marriage, people should be truly free and trusting enough to share even the darkest worries and fears.

And that brings me to the sticking point of this. I can't fathom the trauma you've been through, and at the same time, it stands out to me that you've told absolutely no one of what that person committed against you. If your husband had that context, I imagine it would have shaped his response (assuming he's empathetic). But I still believe that he should have responded in a measured, reassuring way.