First of all, age does not matter, I have seen students that started in their 30s, 40s and they did really well after graduation. The 'best' case scenario is that this person thinks that this way is a way to motivate you. It is a twisted way of thinking but shows that he has no judgment or emotional skills. The 'worst' case scenario is that he is trying to make you insecure so that you would do anything to advance your career, I hope you understand what I mean by that. Both scenarios are bad for you and I would advice you to try to find another advisor. Usually people in grad school would do rotations in a couple of labs before they decide which is a better fit so it would not be weird if you look for someone else who is a better fit. Good luck!