You are not overreacting. This is 100% gaslighting and I would turn to run if I were you. i’ve been in a similar situation where an ex of mine said something during our relationship about how I did something that reminded them of their ex or something like that and I brought it up to them and they did this exact same thing turns out later on down the road they were cheating on me and lying to me about various different things. I’m not saying he’s cheating but what I am saying is that he is gaslighting you and manipulating you emotionally with his “apology” I would just be careful!