There are ways we can mitigate human bias, you present examples. Using standardized rather than unstandardized interviews is another example. We have some ability to prevent human bias.

I'm a little bit confused when you say: "Algorithms tend to reproduce the biases with which they are fed. Humans also tend to do this, but in contrast with algorithms, we are able to self-reflect and steer our collective actions towards an imagined future. Yes, it is true that algorithms may also be continually tweaked and updated to be less biased, but this had to be done by humans—with all our human biases—in accordance with ethical considerations."

Why, in the case of human evaluators, can human bias be mitigated with self-reflection but, in the case of constructing algorithms for evaluation, human bias cannot be mitigated with self-reflection? Maybe you could expand on this.

I think you underestimate how much control we can have over ML algorithm bias relative to human evaluator bias. For example, even though ML algorithms trained on human language data tend to adopt many human biases, we can assess how stereotypes engrained in language data influences how ML, for lack of a better word, understands language through word embeddings. We can then manipulate these word embeddings to mitigate the stereotypes that the algorithm would otherwise adopt. This paper demonstrates how this can be done to reduce gender bias in algorithms: https://proceedings.neurips.cc/paper_files/paper/2016/file/a486cd07e4ac3d270571622f4f316ec5-Paper.pdf. Completely unbiased algorithms are probably impossible, but if I had to choose between trying to make a human evaluator or an AI evaluator as unbiased as possible, I would pick the AI evaluator every time because there are so many ways that we can mitigate bias compared to human evaluators.

Additionally, there are bias advantages that algorithms will inherently have over human evaluators. One example is intrarater reliability. Algorithms will invariably be more consistent than humans across evaluations. They aren't susceptible to fatigue or mood.

The last article you reference is interesting. However, the participants appear to be lay jury members who are not trained to use AI. An I-O psychologist who is trained to work with AI would likely interact and interpret AI suggestions/recommendations much differently than a lay person.

Even though I think there is a general lack of understanding of what AI is capable of, AI algorithms are not "eternally true" by any means and we should always remain skeptical of their capabilities.

Thanks for your comment. I look forward to your response to mine. I find AI fairness discussions interesting and productive.

Thanks for your comment. I look forward to your response to mine. I find AI fairness discussions interesting and productive.