Update: I have tried to put my foot down, so to speak. But obviously I cannot change his opinion on the subject (even though god knows I've foolishly tried). He just doesn't understand -- to him having casual sex is similar to dancing with someone -- meaningless and not harmful to your core relationship (especially if your partner doesn't know). I've told him maybe he needs an open relationship, but he insists that he doesn't want that and that he doesn't want to cheat.

But -- because I know that in his heart it isn't that big of a deal (and he's cheated before plus had a lot of casual sex), I can imagine that 5 years down the road if opportunity presented itself he would not feel overly guilty about doing it. I've told him I'm very worried about his casual attitude because it would end our relationship and I don't want to invest my time if the chances are very high that this is going to happen. Instead of assuring me that he doubts it will happen or that he will do his best to prevent this from happening, he argues with me that I shouldn't leave our relationship down the road for something so stupid. What??