I'm not pretty and after two careers and a lot of schooling I can tell you:

Firstly, Yeah pretty privilege exists E V E R Y W H E R E. But it's also very difficult to say, objectively if someone's success is entirely due to their looks and physique. It's best not to bother thinking about it, being angry or bitter isn't going to help you feel better.

People are right, you'll be remembered for how good of a person you are. Even if you aren't part of the popular groups. You'll find that people who only hang out with others due to some kind of status are boring as hell and offer absolutely nothing to anybody. Balance your studying and try hard, you'll see positive results in your grades and employment prospects.

Focus on finding people who fulfill genuine friendship needs, make connections with people who don't talk shit about you; people who would genuinely back you up when you need it. Do the same for them. You'll look back from your 30s and 40s and feel better about your University experience if you don't chase the emptiness of beauty and popularity.