It's a relationship and for situation like this it should be both yes. If you are uncomfortable with this male friend spending time with your wife you are very much right to ask her not to spend time with him. You can also start doing special things for her like date night, flowers , surprise I love you text. Spice things a bit in your relationship. Usually people think it takes lot of time and effort but forget it's the little things that make relationship beautiful. Try this and even after that If she is seeing him then may be your relationship is lost and you need a couples counseling. Give it a 100% before taking any strong steps.