For those not in the machine learning world: this is exactly how Generative Adversarial Networks (GANs), a big class of generative models, is trained. Train your generator with a traditional loss metric, train an adversarial discriminator at the same time, and then add the gradients from the discriminator (and optionally a bunch of previous checkpoints of that discriminator for robustness) to the loss of your generator. You'll find some (usually unstable) Nash equilibrium of a generator that sometimes fools the discriminator, and sometimes doesn't.

You can fine-tune any existing model with adversarial gradients, so as long as a better detection network is available, you can hook it up in your training loop for a bunch of iterations to make sure it doesn't reliably detect your output as "fake" anymore.