Article
Online Hate Speech, AI, and Media
Literacy
How is generative AI used to create hate speech, and how can it be harnessed to detect harmful online content? In this interview, Dr. Ethan Roberts explores the challenges of online hate speech in the age of Artificial Intelligence.
Authors: The Norwegian Center for Holocaust and Minority Studies and The Kaplan Centre at the University of Cape Town
Published: January 2025
Online hate speech and AI
The advent of AI has been one of the biggest developments over the last decade,but with these developments come a huge number of challenges for online hate speech in particular. One of the biggest points is that people can use generative models to automatically create content that is either hateful or misleading, and deciphering misinformation from genuine information is a huge challenge. The ease with which one can use generative models to create either hateful or misleading content has grown massively, and people can now do this with very little effort at a very large scale.
Additionally, the algorithms that are used to detect this kind of content are subject to bias, and mitigating this bias is actually very difficult. However, AI is actually one of our greatest tools now in identifying hate speech, and we’re able to identify both hate speech and misinformation more effectively than we’ve ever been able to before. Machine learning these days is a catch-all term that people use to encapsulate anything from AI to just plain old statistics.
Generally when people speak about machine learning, they’re speaking about a family of predictive algorithms, and these algorithms generate predictions by learning from data. So we found that large language models are actually very good at picking up hate speech, and are a great tool to add to our toolbox. Furthermore, we find that contextual information is actually incredibly important, and solutions that they develop in the global north don’t necessarily apply everywhere.
So we found that it’s incredibly important to have contextual and nuanced data from which to train. From a linguistic point of view, South Africa is unique in that it has so many languages, and people engage in a multilingual manner online. And the use of multilingual slang is actually something that AI still struggles with today.
This is one of the main challenges in accurately identifying hate speech. So an important research gap in the field of online hate speech is that people don’t have good digital media literacy, and actually educating people on how to be robust against both consuming hateful content and believing misinformation is an incredibly important step.
