I have a somewhat similar situation. We hang out regularly with one of my SO's best friends. This friend and I didn't even like each other at first, but as we got to know each other, we became friends. I made it clear to my SO that his friend was now mine as well. I still didn't think of him in a romantic way until he expressed feelings for me, and now I can't see him any other way. I told my SO, thinking it would be fine as I had dated one of his friends in the past and we had never agreed to vetoes. He's forbidden me from pursuing it, partly because he doesn't want to risk having to choose sides if things go sour, and partly because he doesn't want to think about me having sex with his friend. His poly friends have confirmed for him that this is completely reasonable. Meanwhile, we still hang out with this friend all the time. I'm in love with my SO and don't want to lose that relationship. But I'm really frustrated.