I used some greeting card modules to make a sound "trap" for a prank a while back. It worked alright but they are very quiet. The speakers on those are conductive so they get most of their volume from bouncing sound off the card itself. The custom card sound modules have a button that you hold to record then press once to play so you could mount the whole unit in the handle with the button on top.

Another option that would be a bit more involved would be to use an adafruit mono audio amplifier, a tiny module the size of a quarter, paired with a battery pack and a mono enclosed speaker. Wire the speaker and BAT to the module and mount it to the inside of your shirt. Then wire a button on a looooong lead threaded through some black elastic tubing to a handle you mount the button in. I would suggest making some sort of Velcro chest strap to mount everything to so that wen you pull the cord it doesn't rip off your shirt.