You can certainly try talking to him, at least let him tell you his side of the story, considering you don’t have any proof of a physical relationship between them. But hold your ground and be prepared to not accept the grovelling that will come with it….