2020 Rationalist.org — Evolution Tree content
If left unchecked, we could be facing a biased AI.
If AI were running the world, it wouldn't be so bad. AI could make objective judgments on things we, as humans, are prone to messing up. But if we're going to give AI the reins, it has to be fair. That hasn't been the case so far. AI trained on datasets disclosed or edited by humans can also learn those humans' racist, sexist, or other prejudiced tendencies.
Although slowly, programmers seem to be correcting these biases. But even if we manage to keep our own biases out of our code, AI can now improve everything on its own.
A newly published study shows that a network of AI agents can develop both in-group preferences toward similar agents and active biases toward different ones on their own. In fact, scientists concluded that relatively little consciousness is needed to develop these biases, meaning they can easily create them.
AI agents were designed to donate virtual currency to other agents in exchange for something in return. Essentially, they had to choose how much to share and with whom to share. Researchers found two trends: AI agents were more inclined to donate to those with similar characteristics, while the situation was different for those who exhibited active bias. The AI agents appeared to have learned that donating within a group yielded a greater return, and that actively donating to others resulted in a loss for them.
In our world, where algorithms are used specifically to oppress and target Black people, this is a rather abstract form of prejudice we encounter. AI agents are not incapable of developing any kind of contempt toward a particular minority group, as some individuals might. Instead, they harbor a prejudice toward the other, and this prejudice is limited to this specific simulation.
However, the research has significant implications for real-world applications. If left unchecked, algorithms like these could lead to more institutionalized racism and sexism, and perhaps even misanthropic bias.
According to the article, there are quite a few ways to correct this. For example, researchers found that they could reduce levels of bias by forcing AI agents into what they call global learning, or by having them interact with different AI agents outside their own environment. When populations of AI agents generally had more traits, levels of bias decreased because there was more diversity. The researchers also drew a parallel idea: exposing biased people in real life to other perspectives.
These factors reflect the plurality of society; its susceptibility to issues beyond the individual, such as welfare policy, government, historical conflict, culture, religion, and media.
This means we can't just leave AI to its own devices and assume everything will be fine. In her book, "Hello World: Being Human in the Age of Algorithms," published in The Wall Street Journal, mathematician Hannah Fry urges us to be more critical and skeptical of the algorithms that shape our lives. This, as research shows, is not bad advice.
If we want to believe that artificial intelligence systems are objective, unbiased machines, we must accept that there can always be flaws in the system. For now, this means being mindful of ourselves and the algorithms.
Translation: Ayşe Ege Palaz
References
1. Futurism, “Left Unchecked, Artificial Intelligence Can Become Prejudiced All On Its Own” < https://futurism.com/artificial-intelligence-prejudiced/ >
Originally published on Medium on March 02, 2025. English translation of “Yapay Zeka ve Önyargı”.