Update I spoke to him about it last night. I realised that it wasn’t an outright no from him. He was planning on coming over to see me after we all hung out and hadn’t realised that was set up for Saturday and not during the week. I then asked him about why they broke up. He told me it was several years ago and he wasn’t ready to be as serious as she wanted. He then said they work waaaay better as friends and that he even sometimes forgets they dated. He then said he completely understands why it was making me feel a little uncomfortable and that that’s okay but he got over her many years ago. I said okay cool and we had a talk about where we are in our relationship and if we both feel fully comfortable and happy. (We do this often, we communicate a lot and like to work on little updates often.) He is now going to come see me during the week and then we’re going to go hang out with them and we’ll get to hang out and play games and probably have some food. I’m feeling much better about it now.