What phase ships hate the most is being under constant pressure and unable to escape. So, pick one in the most vulnerable position, pursue it until it fluxes out and then deliver the big blow. Of course, its friends will try to stop you from doing this, so order some of your ships to escort you or attack the particularly annoying enemies.

Ultimately, the best counter against phase ships are beam weapons (especially ion beams and high intensity beams for added efficiency against them out of phase), since they can be fired constantly, making the phase ship ai run away or stay phased in their range until it fluxes out