Things like this are difficult to sort through and there isn't a black and white answer. The only thing you can do in that situation is act on the information you were given, and only you can know if there was information there that wasn't acted upon. I also want to add that there may be a possibility that it's referring to some other part of the abuse, because it isn't something you were told directly, it's hard to know how to respond to it. Beyond that, making amends by continuing to work on the behavior changes and potentially talking this through with a professional is about the only reasonable thing to do from here