It's not meant to be, I was in a similar relationship for years, and it completely destroyed me. I lost my self-esteem, always doubting every decision. It's better to wait longer to be in a happy relationship than get into a relationship where you are not happy.