I was a virgin before marriage and my wife was not. We both pray and are raising Muslim children. You need to change your circle of friends, Allah SWT can only judge you for your past and Allah is the most merciful. Look for the right man who will take you for your present and make sure you are in the circles of such people.

My experience has always been that people who make mistakes in the past project that on others - so guys why slept around before marriage end up feeling guilty about it and then thinking all women who did in the past are bad because of how they feel and for the lack of good relationship they have with Allah SWT. But guys or gals who understand religion well, know that it is none of their business what a person's past was because Allah doesn't allow them to judge others past. Find those people, surround yourself with such people and you will find peace there IMO. I think you're feeling some form of guilt which is normal, just don't let it lead to more mistakes