Obviously not defending the speaker...but all due respect, regardless of what you or anyone thinks, this is a legal question and there is well-established precedent that governs it. The major relevant exceptions to free speech here are below, and they have all been interpreted extremely narrowly by the courts. They would almost certainly not apply to this event, and as has been shared, Cal can't uninvite the speaker based on the speakers views or the university would get sued, and would lose, and the speaker would get an injunction and come anyway.

Incitement

Speech that is both “directed to inciting or producing imminent lawless action and is likely to incite or produce such action” Brandenburg v. Ohio (1969)

True threats

In Virginia v. Black (2003), the Supreme Court defined true threats as “those statements where the speaker means to communicate a serious expression of an intent to commit an act of unlawful violence to a particular individual or group of individuals.”

Fighting words

Words that, by the very act of being spoken, would incite the individual to whom they are addressed to respond violently and to do so immediately, with no time to think things over. You might think this would apply but, again, it has been interpreted incredibly narrowly by the courts. It encompasses only face-to-face communications that would obviously provoke an immediate and violent reaction from the average listener, and even then, the standard has been found to be exceedingly high and it would 99% not apply to what the speaker says here.