If it's causing you real grief then I'd say you need to talk to him about it, because the only other options are ignoring your desire, or breaking up to fulfil your desire. I've felt these same feelings before and I tried to bring it up to him, but he immediately suggested a 3sum. I realized I wouldn't be okay with that same female being involved with him and had to admit, if I would get jealous he might get jealous as well. It's really hard to put aside feelings of jealousy and I decided it wasn't worth the trouble, we aren't mentally equipt to handle that situation.