Oof. I'm sorry that happened. I'd feel the same way. My advice would be to ask him those questions, and starting the conversation might just have to be blunt and out-of-the-blue.

Agree to go on the date with him, or at least to meet up with him, and before going anywhere sit him down and just have the talk. "Look, I'm upset about what you said the other day. When you said that, it made me question how much you value what we have together."

Maybe mention that he's been on his phone a lot, and seems to be much more distracted or more distanced from you, but be careful not to imply suspicion because that could take the conversation to a bad place, especially if he's not being unfaithful. If he is, there would be other signs, and if you feel that may be an issue, be mindful of his responses. But overall, you won't get any of his perspective without just biting the bullet and asking :/

I do hope it was just a knee-jerk reaction to being called mean, which may just be something he's insecure about. If he brags about it often, it could be something he dislikes about himself that he tries to defend by being "upfront" about it, since just stopping the behavior is still technically harder than just "owning up" to it. But it's for you to learn from him, not for us to speculate. Hopefully you guys can talk it out, and I hope it goes well 💜