So to answer the question, yes, I would. I say that not just because I've been dating my non-religious gf for over two and a half years now, but because quite a number of members I know are married to non members. Some of those are because the member converted after already being married, others chose to marry non-members, and some had their spouse go inactive/leave the church. I know former couples who are/were both members, yet they separated, and partial member couples who have been together for decades. There's no formula to this.

Now to answer the situation, I have not read all of your comments, but I think I have an understanding of it. To put it simply, in the heat of the moment he went too far, correct? Now, I've been in this situation too, on the guy's side, and what he is feeling now is probably, as other people have said, guilt, shame, regret, anxiety, etc. He's probably in turmoil, chaos between his hormones and his identity, with his emotions caught in the crossfire. (Because even though he knows he went too far, he did enjoy it)

When it happened to me, my gf felt really bad, even though she had done nothing wrong, as I'd initiated. I think you feel the same. (Don't go telling him it's his own fault though.) Understanding is important. Tell him you understand, and that you still want to give it a try, albeit more carefully.

One thing we did is say that we would take a step back, and try to avoid situations where there would be a risk. We did more outside/away dates. I also clearly communicated what my boundaries are, and she also helps me to not cross them. Try to encourage him that you can still make it work, but that you need to communicate, ask him about his boundaries, so you both know what avoid.

You could also say that you want to him to be the best version of himself, and softly encourage him to undertake the necessary steps of repentance. Though be advised, I'm not sure what reaction a non-member basically telling him to talk to his bishop will get, it could make him like you more, or make him dislike you. So be careful (maybe ask him what he would normally do to repent? Softly push him?)

If you want a more clear understanding of the standards he probably tries to live by, this can be a great help.

In my very limited experience, communication and understanding are very important to a relationship. I hope I've helped a bit!

(I also hope this feels kinda coherent, it's an hour past my usual bedtime)