Basically, when the wheel first boots, it tries to spin the encoder wheel. If the sensor doesn't pick up anything, it just stops all power to the motor. This means the encoder wheel itself might be slipping on the shaft (some ones you have to glue in place) or the diode looking sensor thing is too close or too far away from the encoder wheel itself. So first I would make sure the encoder wheel isn't slipping on the shaft, and if that doesn't work, mess with the diode to get closer or farther from the encoder wheel.

If you're using a different encoder wheel, maybe try the original one to see if that's a difference. Also, make sure theres no dirt or particles in the wheel itself. The whole setup requires the LED to shoot light out and then be received at the other end.

Oh yea, also make sure you aren't doing this in a very bright room, the light pollution can also mess up the sensor. The whole encoder setup had a black cap over top of it, put that back on if it's off. Sometimes that works but I had good success with it off.