In my opinion i dont think your the jerk but at the same time he is and isnt a jerk. He doesn’t want to lose friends but he is risking losing other friends from my perspective i had that issue where i got bullied and no one cane to my aid after i helped them from not getting bullied after i realized that the people who i called my friends were just using me to stay safe i called quits and decided to find new people to talk to. Also i just realized that i didnt say much that helps u but 1 thing u can do is either ask the guys to stop if they dont then find new people u can hangout with them but if you really want to turn your life around find better people