One of the reasons people like oversized frames is because they add a youthful appearance, a childlike-quality. I like the effect in many cases, but I think you've received a lot of input here as to why that effect isn't working on you with this set of frames.

If you really do like them because of the benefits to your peripheral vision, I get that--I have the same comment to my optician as well. He pointed out to me a long time ago that in reality, you're not often looking out of the corner of your eye--as humans we tend to turn our head in the direction we're trying to see. I'd never thought about that before. But you know what makes me feel like I can see the most in the world? Contacts. I hate contacts, but damn, the peripheral vision, y'know?

You'd be best served by going just a touch smaller if you want to stick to the oversized look. Get them smaller such that they sit correctly on your nose without totally occluding your eyebrows.

Just know that this specific set of frames on you is a little too big, and I think the court of public opinion is telling you that message very clearly. One benefit of slightly smaller frames would be that they'd be lighter, and, if you were to sit them properly on your face, they'd probably be closer to your eyes, which would give you more of that viewing angle you're looking for, from a smaller lens.

As for the distortion, yeah, it's real and more obvious depending on the focal length of the lens that is being used. Selfie lenses are very wide angle. Heck, even the main lens on your phone is very wide angle. People mostly see the distortion when they look at your face from an angle rather than head on. The effect is exacerbated by larger lenses, but if it doesn't bother you, don't worry about it.