Special report MIT has taken offline its highly cited dataset that trained AI systems to potentially describe people using racist, misogynistic, and other problematic terms.
The database was removed this week after The Register alerted the American super-college. MIT also urged researchers and developers to stop using the training library, and to delete any copies. “We sincerely apologize,” a professor told us.
The training set, built by the university, has been used to teach machine-learning models to automatically identify and list the people and objects depicted in still images. For example, if you show one of these systems a photo of a park, it might tell you about the children, adults, pets, picnic spreads, grass, and trees present in the snap. Thanks to MIT’s cavalier approach when assembling its training set, though, these systems may also label women as whores or bitches, and Black and Asian people with derogatory language. The database also contained close-up pictures of female genitalia labeled with the C-word.
On one level, this is sort of hilarious. How do you get a massive dataset with that sort of information in it? Did some rogue programmer at MIT seed dirty words into the data? Who in their right mind would do this?
And the answer of course is human beings. No one at MIT intentionally coded these words into the dataset. Instead, when MIT programmed their computers to search the web for images and associated descriptors, they likely did so with the notion that accuracy was the most important criterion. That’s not wholly unreasonable.
Except that the truth is human beings use derogatory terms. Some people use them frequently. MIT’s computers didn’t have a bad language filter so what went into the dataset was highly accurate in the way that human beings, collectively, use language.
As it turns out, that’s not what MIT or other researchers or developers wanted. What they really wanted was a dataset that was not accurate in the way that people use language but how people should use language.
Depending on the ultimate use of the data in the build new applications, eliminating offensive associations could be appropriate. But it’s one thing to avoid propagating these associations (good) and another thing altogether not to recognize that they exist (bad). And the question of what is actually offensive has bedeviled humanity for most of recorded history, so who decides what’s offensive is rather fraught with danger.
If you’re trying to build the best Artificial Intelligence (AI) you can, it has to understand the association with words in the ways that people use them even if we don’t like how they use them. It doesn’t mean that the AI has to use those terms itself as far as output, but—like we all do—it should understand the meaning of the words.
Here is one example. If a person has captioned an image, “Me hanging with my bitches” it’s unhelpful if an AI responds with, “I don’t see any female dogs in that image” unless there really are female dogs in the picture. Although there is a temptation to say, “No, it is helpful. It’s a teachable moment” that’s not how it will play out unless the AI understands the association in the first place.
Indeed, only if the AI does understand that it’s an offensive association, could it become a teachable moment.