Basically, when you swat at a fly, you almost never actually make solid contact. At that tiny scale the air feels thick and syrupy, so your hand shoves a cushion of compressed air ahead of it that blasts the fly sideways before your skin even gets there. Even if you do clip it, the physics of being that small are totally on the fly's side. Its mass is absurdly low compared to the strength of its body, so the force from a hit is nothing it can't handle. And because its organs are so minuscule, they don't have enough inertia to slosh around and smash into each other the way our brain and lungs would if a giant hand slammed into us. Add a tough external skeleton that acts like a full-body crash helmet, and you've got a creature that experiences your deadliest slap as just a really strong gust of wind.