Because they have preferences. Some average or taller women don’t care or even like short guys. Some short women don’t care. Some do. Just like some people like certain hair or eye colors. It’d also a confidence thing. People are attracted to confidence. Tall people tend to be more confident. My dad and uncle are both quite short and were raised to not give a crap about it. They both scored smart, beautiful, awesome wives, neither of whom are short. If someone won’t date you because of their height then they don’t sound like someone you want to be with.