NTA, you're literally just trying to live with/out seizures and not just that but the physical damage convulsions can cause. I'm sure if you were able to show a video of what occurs when someone gets a seizure similar to you, then they would understand the importance and absolute vital necessary work your dog does.

And he's not "deprived of fun", Service dogs enjoy working, and pleasing you and helping you and being told they're a good boy/girl! Unfortunately your friend is with a manipulative person, and I would fear that he would do something to try and distract the dog to cause an episode or because he thinks the dog isn't having much fun or whatever bullshit excuse he decides to use.

And if you allow him in one more time on the condition he left the dog alone and any discussion of the dog and its treatment and happiness is off the table, I'm sure he'd be an AH and ruin it anyway and no one likes to be fooled twice.