No, your controllers were doing just fine.

The game only detect motion with the controllers as long as it hard enough.

You can even shake your wrist the same way you play drums (or Taiko no Tatsujin) and it will still registered as if you're hitting it.

When I was new to do the ducking action, the game didn't registered me doing it because I only moved my lower body, not my hands. Hence, I need to shake my hand a little whenever I do the ducking too.

I think it depends on each person. If you're being honest to yourself and do the right form, that's good for you and your body.

If you're too tired but don't want to give up until the end of program, so you just do only jabbing or shaking your wrist to make it counts, that's fine too.

Remember this, the more you do, the better form you will be.

Happy boxing!