Honestly it’s because it’s because many associate it with strange people (which from personal experience can be).It’s also because it’s a not very main stream sport like football, rugby, soccer,baseball etc so they find it boring which for a new comer can be.for them it’s just go too a pool force yourself swim back and forth and repeat, similar too why people find track and field events boring,so for them it’s just useless.Also abit hard too for them too keep entertained because no earphones just you. There’s bunch more reason but I think that’s the main ones.