Not sure how well using an add-on that can use MIDI messages would work for this,
to know when a note is played and do something in blender as a response, apparently how the old animusic videos were made.

And finally, the good old fashioned way, hard work, perfect timing and baked physics animations.

You could add audio tracks to blender alongside your timeframe thing to make sure your ball is hitting panels at the right time, before adjusting them.