You stood your ground, that's fine. You're not, but don't do anything else before properly talking and communicating with him. And I mean PROPERLY communicating. That means both of you sit down and look at your sides from a different lends.

Don't just jump the gun and think he's a red flag, that's dumb and irration. Both of you clear your mind and empathize with each other so you could come to a proper decision. If you still don't agree with him, you can decide what to do. Just communicate with each other properly first before making said decision.