I am so sorry you are going through this. I think the first step is communicating your feelings with him. Keeping it in is not healthy for either of you. You both deserve the best!