![\"\"](\"https:\/\/world.edu\/wp-content\/uploads\/2023\/05\/Judged-by-Bias-AI.jpg\")
<\/p>\n<\/p>\n
In 1998, I unintentionally created a racially biased artificial intelligence algorithm. There are lessons in that story that resonate even more strongly today.<\/p>\n
The dangers of\u00a0bias and errors in AI algorithms<\/a>\u00a0are now well known. Why, then, has there been a flurry of blunders by tech companies in recent months, especially in the world of AI chatbots and image generators? Initial versions of ChatGPT produced\u00a0racist output<\/a>. The DALL-E 2 and Stable Diffusion image generators both showed\u00a0racial bias<\/a>\u00a0in the pictures they created.<\/p>\n My own epiphany as a white male\u00a0computer scientist<\/a>\u00a0occurred while teaching a computer science class in 2021. The class had just viewed a video poem by Joy Buolamwini,\u00a0AI researcher and artist<\/a>\u00a0and the self-described\u00a0poet of code<\/a>. Her 2019 video poem \u201cAI, Ain\u2019t I a Woman?<\/a>\u201d is a devastating three-minute expos\u00e9 of racial and gender biases in automatic face recognition systems \u2013 systems developed by tech companies like Google and Microsoft.<\/p>\n The systems often fail on women of color, incorrectly labeling them as male. Some of the failures are particularly egregious: The hair of Black civil rights leader Ida B. Wells is labeled as a \u201ccoonskin cap\u201d; another Black woman is labeled as possessing a \u201cwalrus mustache.\u201d<\/p>\n I had a horrible d\u00e9j\u00e0 vu moment in that computer science class: I suddenly remembered that I, too, had once created a racially biased algorithm. In 1998, I was a doctoral student. My project involved tracking the movements of a person\u2019s head based on input from a video camera. My doctoral adviser had already developed\u00a0mathematical techniques<\/a>\u00a0for accurately following the head in certain situations, but the system needed to be much faster and more robust. Earlier in the 1990s,\u00a0researchers in other labs<\/a>\u00a0had shown that skin-colored areas of an image could be extracted in real time. So we decided to focus on skin color as an additional cue for the tracker.<\/p>\n I used a digital camera \u2013 still a rarity at that time \u2013 to take a few shots of my own hand and face, and I also snapped the hands and faces of two or three other people who happened to be in the building. It was easy to manually extract some of the skin-colored pixels from these images and construct a statistical model for the skin colors. After some tweaking and debugging, we had a surprisingly robust real-time\u00a0head-tracking system<\/a>.<\/p>\n Not long afterward, my adviser asked me to demonstrate the system to some visiting company executives. When they walked into the room, I was instantly flooded with anxiety: the executives were Japanese. In my casual experiment to see if a simple statistical model would work with our prototype, I had collected data from myself and a handful of others who happened to be in the building. But 100% of these subjects had \u201cwhite\u201d skin; the Japanese executives did not.<\/p>\n Miraculously, the system worked reasonably well on the executives anyway. But I was shocked by the realization that I had created a racially biased system that could have easily failed for other nonwhite people.<\/p>\n How and why do well-educated, well-intentioned scientists produce biased AI systems? Sociological theories of privilege provide one useful lens.<\/p>\n Ten years before I created the head-tracking system, the scholar Peggy McIntosh proposed the idea of an \u201cinvisible knapsack<\/a>\u201d carried around by white people. Inside the knapsack is a treasure trove of privileges such as \u201cI can do well in a challenging situation without being called a credit to my race,\u201d and \u201cI can criticize our government and talk about how much I fear its policies and behavior without being seen as a cultural outsider.\u201d<\/p>\n In the age of AI, that knapsack needs some new items, such as \u201cAI systems won\u2019t give poor results because of my race.\u201d The invisible knapsack of a white scientist would also need: \u201cI can develop an AI system based on my own appearance, and know it will work well for most of my users.\u201dEchoing through the years<\/strong><\/h4>\n
Privilege and priorities<\/strong><\/h4>\n
\n