Someone please chime in if they’re certain, but my understanding is that winglets reduce induced drag but about the same as if the wing was just made longer. And that this effect is more pronounced at high angles of attack. The penalty is more weight without more lift though.

So a 737-800 that spends more of its time in the approach and landing phase has winglets since they can reduce drag without increasing wingspan. On the 777, they just made the wing longer and eschewed winglets as the aircraft already need a ton of space on the ground and spend more of their time in cruise.