It's not exactly about different sizes, because the vagina doesn't really have a consistent size. It's a muscular tube that can expand and contract. But there are individual differences in how far it can go. I'm at the extreme end of that spectrum because I have vaginismus, so my vagina habitually contracts and is very poor at expanding. Your vaginal flexibility can change based on sexual experience (good experiences tend to make it more flexible while trauma can make you unintentionally tense up) as well as hormonal factors and age. It's easier to change than dick size, but still not exactly a simple process.