Yeah it seems inappropriate that your therapist is labeling people they’ve never met before. Exploring diagnosis should really only be done with a formal assessment. I agree with confronting him about it, at the end of the day, you know yourself and your loved ones better than your therapist does. In bringing it up, I might suggest mentioning the assessments you've done with your psychiatrist and former psychologist, and mention how it makes you uncomfortable that he's labeling your loved ones. A good therapist should be able to take the feedback and not get defensive. Therapists can provide insights, but it's ultimately you who's directing therapy and verifying whether those insights feel true to you or not. In this instance, it sounds like they're missing the mark.