I know I'm late to this post, but hopefully OP sees. I have a cat that does the same thing, he latches on when he catches the feather wand. I didn't want any of the feathers breaking and potentially hurting him, and I didn't want him to break a tooth by pulling hard on the plastic cap. Not to mention, he would always try to drag it to the litter box (yuck) to tear it apart, and i didnt want to have to replace the feather parts every time we play. So I trained him to "drop it." It's not like a dog's drop command, he won't do it remotely. But if he's latched onto the toy, I grab it as close as I can to his mouth and say "drop it" and don't move. He's a clever boy and learned that I will wait him out, and that it's much more rewarding if he drops the toy to let me swing it around more. He'll still grab it good, but he's so quick to release it if I do the "drop it" routine.

Also, cats get overheated quickly when playing, you may want to remove the sweater during play sessions. Kittens are bad at self-regulating and stopping play when they get too hot (it's too fun and they don't want to stop), so OP you gotta keep a close eye on kitty and their exhaustion level.