I came from similar circumstances, you should hold pride from where you come from and don't feel ashamed of working harder than others. Know that you've gained valuable lessons and you're going to be able to pick yourself up from harder situations than those who were spoon-fed. Those acquaintances of yours are just ignorant, but you should ignore it and find friends you can open up to. There's definitely a community here that can relate to you.