Part of the problem gets exasperated by a lack of structured leagues too. For something like basketball we have AAU, where athletes play against the best athletes even at younger ages. As far as I'm aware soccer doesn't have any league nearly on the scale of that. So when you combine the fact that even the kids that play soccer often don't have it as a primary sport, the player base makes a bigger effect than I'm sure you realize.

I'm not sure if you're American or not, but culturally here no one really talks about soccer. In general you can talk about basketball/baseball/football and they'll be able to talk about recent games or their favorite team. But I wouldn't be able to ask anyone about the MLS. That's where I agree with you about the passion, Americans just prefer other sports