A disabled automatic photo-cropping system by Twitter discriminates on the basis of color, gender, weight, and age, according to the results of an open competition to find algorithmic bias, media reports said.
The microblogging site had, in March, disabled the system as users noted that it favored white and female faces while auto-cropping images. The Verge reported that it then launched an algorithmic bug bounty competition offering prizes of up to $3,500 to analyze how the technology incorrectly handles photos.
The top entry, contributed by Bogdan Kulynych, a graduate student in computer security at EPFL in Switzerland, showed that Twitter’s cropping algorithm favors faces that are “slim, young, of light or warm skin color and smooth skin texture, and with stereotypically feminine facial traits”.
These algorithmic biases amplify biases in society, literally cropping out “those who do not meet the algorithm’s preferences of body weight, age, skin color”, Kulynych noted in his summary.
The report said that the second and third-placed entries showed that the system was biased against people with white or grey hair, suggesting age discrimination, and prefers English over Arabic script in images.
“When we think about biases in our models, it’s not just about the academic or the experimental … but how that also works with the way we think in society,” Rumman Chowdhury, director of Twitter’s META team (which studies Machine learning Ethics, Transparency, and Accountability) was quoted as saying at the DEF CON 29 conference while presenting the results.
“I use the phrase ‘life imitating art imitating life’. We create these filters because we think that’s what beautiful is, and that ends up training our models and driving these unrealistic notions of what it means to be attractive,” he added.
Twitter’s open approach is a contrast to the responses from other tech companies when confronted with similar problems.
“The ability of folks entering a competition like this to deep dive into a particular type of harm or bias is something that teams in corporations don’t have the luxury to do,” Chowdhury said.
According to Patrick Hall, a judge in Twitter’s competition and an AI researcher working in algorithmic discrimination, biases exist in all AI systems but companies must work proactively to find them.
