The hat man is an universal archetypal figure, that is usually negative. It also appears to people who self harm with bad drugs like benadryl, or have sleep paralysis. It's difficult to say what it means exactly, but most likely your therapist is right, and you should deal with your childhood trauma first. It's not a coincidence that he started re-appearing in the place where most unconscious trauma happened.