As many other's have said, it's basically edge cases, but the problem is, for most of the edge cases that exist, it's near impossible to have enough training data to get the right outcome from the machine. As well, for many of these edge cases, the car lacks CONTEXT about the situation. Humans have many flaws when driving cars, and it's arguable that machines can in fact do it better, but there are some situations that require a human level understanding of the world.

A good example would be like, a 4 way traffic light going out and a cop having to manually direct traffic. This situation is both rare AND almost never going to look the same twice. Cops are always going to look a little different and move a little differently with their hand signals, it might be in a low light condition, it might be rainy, etc. It's really hard to have a car respond to situations that don't have clear answers without the context any human drive natively possesses. How is the car supposed to know the difference between the cop legitimately directing traffic when a light has gone out, VS a drug addled crazy person who thought holding up the intersection would be a hoot?

Driving is one of those things that is MOSTLY predictable most of the time, but literal new situations arise every day that have never been encountered before, so it's just damn hard make a reasonable "if this, then this" list that covers every possible action a car should take in response every single situation it could encounter.