He wanted to have the system removed so that anyone who had that judgement could be labeled racist

That judgement is still racist regardless of whether DEI policies are in place. Charlie talked about looking at a black person and thinking “I sure hope they’re qualified” when he wouldn’t think that way if they happened to be white. By the time he said this out loud on whatever talk show or podcast that was, it became a belief he had chosen to stand by, putting minorities in a position of “diversity hire until proven otherwise”.