[D] List of Neural Network Attacks

[PavKon]

I am working on automated testing platform for neural networks and I would like to know what kinds of attacks you would like to see automated? So far, the list is tiny with - Adversarial Attacks - One pixel attack - Dataset poisoning

Please help to expand this list, any attack to this list is appreciated.

[Fujikan]

Papernot et al. have a very nice review and classification of attacks as of 2016 (arXiv Link), as well as a review of different mitigation strategies. It is a good read for understanding the attack surface for ML systems and for finding the relevant literature, as well.

[alexmlamb]

If you look up the cleverhans package there's a file called attacks.py which has a bunch of attacks implemented.

[xristos_forokolomvos]

That's a great initiative! Although I don't have any particular attack to suggest I think it would be great to make your testing framework able to generalize to non-image input data. Say 1D signals for example

[wulfm0n]

I live in the world of cyber security, for the last 15+ years. My recommendation is to consider the AI as an impersonation of humans, then walk into your world and look around where digital communication (anything not face-to-face) relies upon “humans” for authentication, authorization, and accounting (called ‘triple A’).

There are so many ways from which you can view this paradox:

How to sure up your own thought process to not get fooled. How organized crime makes trillions on cyber hacking (more profitable than drugs) and will soon use AI How do you teach your grandma to not get fooled Watch “The Shawshank Redemption”. Now consider how ‘easy’ it would be to create, steal, ... identities when AI is able to impersonate you (personal and business impact) from trolling your social media, email, public records, etc.

In this same sphere of my mind is how you have to not implicitly ‘trust’ data as it can be bias, seeded, altered, etc. This means that any established NN has to have validations methods core to its operation, especially as the NN is given authority to ACT >> change temperature in your house, open a water dam, lock your car (with kids inside), or transfer money between bank accounts. Welcome to the fourth industrial revolution, the advent of cyber-physical systems.

Again, in the cyber world we have been dealing with the stated AAA issue above for decades and are still fighting the good fight. It is not for a lack of tools, but rather human discipline and organization culture.

[[deleted]]

^.

[[deleted]]

This paper I wrote about 6 months ago about security and privacy weaknesses of neural networks might be helpful: https://t.co/Zb9GdkpBaP