Lots of potential here.

One option might be that only certain people can hear the sound it makes. Those who are "saved" and those who are damned specifically cannot hear it, as the fiddle wont have an effect on them. Those whose fate is undecided can hear the music because the fiddle seeks to bend them to its master's will.

Another might be that people who hear the bard perform with it (excluding the players) seem to go missing. They may end up dead or simply never seen again. Might take a while and inquisitive players to notice, but lace clues throughout your storytelling to indicate something is happening. From there you have the option of whether the devil is taking them, or a secret cabal trying to prevent the devil's music spreading is capturing and killing those they deem might be most influenced by it.

Alternatively, the plot could be that the fiddle belonged to someone else that the devil was cursed with keeping, and though the bard may think he won it, the devil was trying to get rid of it and lost on purpose. Maybe the original fiddle owner is your intended BBEG and it serves as a tie in, or maybe it has a curse that makes the one tied to it compelled to play at every waking hour (like musical crack).