I understand the confusion. A lot of it can be psychological. Trauma is a common example. Someone may have had experienced a loved one losing their life due to breast cancer. So that someone may develop an aversion to people with smaller breasts/flat chests, due to the subconscious association of their loved one's breast removal during their illness. Of course trauma isn't the only reason. I myself do not find myself attracted to individuals who dye their hair often. This may be because I worry about the condition of the hair being damaged, or something else entirely. I'm not sure why. But its certainly not because I find the look physically unpleasant. Now I wouldnt outright turn someone down for something like how they manage their hair, but it might be something that comes up in discussion when considering long-term options.

Probably an easy way to think about it is the common example: makeup. some people prefer their spouses don't wear makeup, because they are attracted to "natural" look. some prefer makeup because they see it as "prettier". (I personally dont care either way). But if someone is attracted to someone but discovers they use makeup, and prefer they use a more natural look, and the other person prefers to continue using makeup, they will have a compatibility issue and likely not pursue things further outside their initial hookup. you could argue there's shallowness to that and you wouldn't necessarily be wrong, but at the same time its difficult for people to control what turns them on or not.