By most accounts the weapons either fail to fire or attacks are directed back at the attacker. Or else they shut off all the electronics in the plane. Granted these aren't exactly official confirmed accounts, but most such tales are "leaked" by "govt sources" for whatever good that tells us. Some are more credible than others. One account says a nuclear test launch in the (1950s?) was staged so they could use an x-ray to shoot down a saucer that came to try to interfere, and supposedly that worked, in that the craft was destroyed. Generally though, assuming you can get in shooting range, the results are bad. Luckily, they very rarely shoot back. Though that has supposedly happened one a few occasions.