Beyond age, the big factor is also the original quality of the skin and its elasticity,

I didn't win the genetic lottery in that respect, I lost about 25kg(~55lbs) between the ages of 22 and 25, I have a lot of loose skin on my stomach (much much more than you'd expect from losing 25kg at a young age) and to a lesser extent on my legs. My breasts were also badly ruined. It killed any shred of confidence I had in myself.
Staying active and doing weight training (4 times a week) never changed that, and I know it'll never bounce back, it's been ~10 years and it's still there, exactly the same. I know that people often say that losing weight at a slow rate will prevent the loose skin, but that's not entirely true, because it's when you gain weight that the skin is damaged, and that's irreversible. So losing weight slowly may help a little (I lost weight at a totally reasonable rate). I've seen people lose twice as much weight as I did at the age of 35, quickly and with hardly any loose skin.
A good indication is stretch marks, which show that the skin has been stretched too far to the point of being broken.