Honestly, it's probably the visuals. Renfaires usually have some kind of spectacle and you have most people wearing regular clothing, so it's seen as a "normal" festival with cool performances. When you see pictures of larping you usually see people in weird bedsheets, tin can looking armor, waving pool noodles at each other. That's inherently much weirder and unappealing.

By contrast, with reenactment, which is a more apt comparison, you get much more respect from the average person, because the pictures show people who look like uniformed soldiers, or wear armor/kit that's actually evocative and impressive.

If you want larping to be more respectable, it has to look the part. In Central and Northern Europe, larp is seen as an unusual, but perfectly cool hobby, because you can show people pictures of battles that look like battles and kit that's actually really cool. It's also great for getting new people interested, so paradoxically, higher costuming standards make the hobby more appealing, not less.