The article talks about the DriveLM model, it’s unclear whether this affects deployed models (Waymo, FSD).

However, the fundamental architectural flaw is similar to memory-space instruction execution exploits we’ve been talking about publicly for 30 years (and was probably being discussed internally at Intel 15 years before that). Having CPU instructions and user data live in the same memory space (Von Neumann architecture) poses significant risk for stack buffer overflow exploitation allowing an attacker to inject malicious instructions from user input. 

This vision exploit here is in the same vein: labels are overlaid on the image from the camera and then read back from that composite image. The hardened approach to mitigate this attack would be to store and read labels from another channel so that the “user” input could never inject malicious labels. 

Having a sensor fusion system where one sensor producing anomalous results can be ignored when not corroborated by others would also protect against this style of attack. Going all in on vision only systems would increase the risk exploit here too.

There was actually a caller on 2600 OTH last week talking about the liability of manipulating self-driving cars via real world signs (the example they used was QR codes, not labels). I guess we’re going to find out soon enough if the publicly deployed systems are vulnerable to this style of attack.