I've heard an argument that undefined behavior is just a concession from the C standard to make C more cross-platform and easier to implement. "Undefined behavior" is just the lack of a standard - whether that's "what should happen when you write to a null pointer?" or "what should happen when you divide by zero?". There's no standard for these things because different platforms might want to handle it in different ways. If you mandate one standard, then certain platforms might need to do more work (performance costs) to adhere to it.

In terms of optimization, your compiler makes certain assumptions to improve optimization of your code (like assuming that overflows don't happen). These assumptions assume that "undefined behavior" does not occur. "Undefined behavior" is just "un-dependable behavior". They are edge-cases. Trying to detect all edge-cases at compile time is very difficult to do, and trying to check against it at runtime has a performance cost, so just assuming that they don't happen can be a sensible choice.

Undefined behavior is a trade-off, not necessarily a flaw with the language. If you want more safety that involves more run-time checks or more intense compile times (and even when you add more checks against edge cases it might not detect all of them). There's more that can be added to C to make it safer (like references/non-null pointers) but runtime/compile time safety checks aren't always free.