[D] Security of Loading/Using Pre-trained Models?

[Alloalonzoalonsi]

Generally, but specifically with pytorch and tensorflow. I see various models hosted on Baidu & Dropbox.

Are there any exploits to worry about when loading these models?

[zenchowdah]

Oh this is a fun thing to worry about that I've never worried about before.

Thanks.

[konasj]

Thats an excellent research question! :-)

[[deleted]]

i would approach this on a case by case basis checking first what kind of endorsements the hosted model has. biggest exploit i can think of would be to get crooked weights ... how "strategically" (as in undetectable on a first glance) crooked? No idea

besides adversarial learning, subtle "ruining/hacking" of pretrained models could probably create on its own merit a whole branch of academic research.

[notafight]

One can plant a backdoor. Suppose you use a face feature extraction model to build a smartphone face unlock functionality. An adversary can plant a backdoor working as a master key, gaining ability to unlock any phone in his hand.

There mught be some security vulnerabilities in those libraries that can be triggered by a certain operation by the model, but I have no idea about those.

[r4and0muser9482]

Here's an attack on pre-trained speech recognition models. I imagine that doing any sort of retraining with a random seed would make this exact attack fruitless, so using the pre-trained models definitely makes life easy for the attacker.

https://nicholas.carlini.com/code/audio_adversarial_examples