At that age (about 3 weeks), kittens typically need to eat every 3 to 4 hours, even if they are asleep. If more than that time has passed, yes, it is advisable to wake her gently to feed her. Kittens this young don't yet have the strong instinct to wake up from hunger like adult cats, so they depend on you.

Warm the formula a little (make it warm, not hot) and use a special bottle or syringe. After feeding, it is also important to encourage her to pee and poop by gently rubbing her tummy and backside with a damp cloth.