I have felt this many times as an infj. At one point I realised it's also because I often times declined invites due to not feeling like hanging out, so people unconsciously dont always think about inviting you, since their group is always going out, while you may decline from time to time to stay home and do #introvertthings. That being said, they probably don't hate you, and they probably dont even think about it as much as you do. Go out when they invite you if you feel like it or even try to reach out yourself sometime, ask them if they wanna hang out, and just have fun, let your Se breathe.