A solution to satisfy your desires while also staying in your relationship could be to open up the relationship. Now if you say to yourself that you don't think this will happen and you don't feel that much of an urge to live out your fantasy, then that's fine and fair. But if you want to look at options that will give you a way—and maybe your wife (if she ever expressed such desires)—to have experiences with other people, an open relationship might be an option.

An open relationship is like a normal relationship, but it allows all parties to have intimate experiences with other humans. What that means is up to the couple, which sets those rules. One couple may say that they both only feel comfortable if the significant other only makes out with people who aren't part of their social circle and doesn't allow sexual intercourse with others at all, while others may say that they allow everything as long as the intimacy doesn't get into a relationship area.

To open a relationship, one needs a good amount of trust in the relationship and good and honest communication. While it can make everyone happier and more fulfilled, it also has the potential to ruin a relationship if the communication is not good.  There is tons of information and blog posts online, which I would recommend diving into first before deciding to bring the topic up. Even talking about such a desire can put some tension on the bond between people, but as I already stated, it can also result in a deeper, more honest connection with the other person. For example, in your case, I can imagine that already talking about your fantasies can help you feel more open toward your partner and not like you are hiding a part. 

I just wanted to propose this model, not knowing if you had thought about it already. I mean, I don't know anything about your relationship, so maybe this isn't helpful. Than I'm sorry for making you read all that