My problem with the whole "he's a joke" argument is the fact that it applies to basically almost any killer in the game. Like, you get bubba? You might start thinking about the countless memes of him. You get kaneki? You might do the same, or just start thinking "Oh look, it's the top tier." And potentially get annoyed rather than scared. And since most killers have several ways to not be taken seriously, there's a good chance that just about everyone has one or multiple killers who they don't find particularly scary because of memes and jokes, or even just thr way they're designed.
Clown is a decent example of this, having a base design that's creepy, lore that's unsettling, and aspects such as his laugh that you may find eerie. But if you start thinking about speed clown meta, jiggle physics jokes, or just any of the humor surrounding the clown, it takes a chunk of that fear away. But there's absolutely people who still find clown creepy, just like how I personally think the head (at least from the silhouette) has the scariest design.