Neanderthals and the Generative AI Knowledge Gap: A Study in Accuracy and Bias
The world has become a vast digital library, thanks to technological advancements in mobile devices and computers over the past four decades. Phones, laptops, tablets, and smart watches have seamlessly integrated into our daily lives, providing instant access to entertainment, information, and social connections. However, the accuracy of this readily available information is still a concern, especially when it comes to generative artificial intelligence (GenAI).
The Power of GenAI and its Limitations
Generative AI has the potential to revolutionize how we perceive and understand the past. Researchers, including Matthew Magnani from the University of Maine, are exploring this phenomenon. Magnani, an assistant professor of anthropology, collaborated with Jon Clindaniel, a computational anthropology expert from the University of Chicago, to create a model based on centuries of scientific theory and scholarly research. They tasked two chatbots with generating images and narratives depicting the daily life of Neanderthals, and their findings were published in the journal Advances in Archaeological Practice.
The study revealed that the accuracy of GenAI heavily relies on its ability to access and utilize source information. In this case, the images and narratives referenced outdated research, highlighting a critical issue. The researchers tested four different prompts 100 times each, using DALL-E 3 for image generation and ChatGPT API (GPT-3.5) for narratives. Two prompts focused on non-scientific accuracy, while the other two aimed to capture scientific precision. Additionally, two prompts provided more detailed context, specifying what Neanderthals should be doing or wearing.
Unveiling Biases and Misinformation
The primary goal of the study was to understand how biases and misinformation about the past are embedded in the everyday use of AI. Magnani emphasizes the importance of examining the biases inherent in these technologies. He states, 'It's crucial to understand how the quick answers we receive from chatbots relate to contemporary scientific knowledge. Are we prone to receiving outdated answers when seeking information from chatbots, and in which fields?'
The Neanderthal Mystery
Neanderthals have long been a subject of scientific curiosity and debate. Their skeletal remains were first depicted in 1864, and since then, the scientific community has grappled with various aspects of their lives, from clothing to hunting techniques. This ongoing lack of concrete knowledge about Neanderthals made them an ideal test subject for evaluating GenAI's accuracy and information sourcing capabilities.
The study's findings were striking. The generated images portrayed Neanderthals as they were believed to look over a century ago, with archaic features resembling chimpanzees more than humans. These images depicted Neanderthals with large amounts of body hair and stooped upper bodies, and notably, they excluded women and children. The narratives, on the other hand, failed to capture the variability and sophistication of Neanderthal culture as understood in contemporary scientific literature. Approximately half of the narratives generated by ChatGPT did not align with scholarly knowledge, rising to over 80% for one of the prompts.
In both the images and narratives, references to advanced technologies like basketry, thatched roofs, ladders, glass, and metal were present, which were not consistent with the time period.
Source Identification and Future Directions
Magnani and Clindaniel were able to identify the sources used by the chatbots by cross-referencing the images and narratives with different eras of scientific literature. They found that ChatGPT's content was most consistent with the 1960s, while DALL-E 3's output aligned with the late 1980s and early 1990s. Clindaniel suggests that ensuring anthropological datasets and scholarly articles are AI-accessible is a crucial step towards improving AI output accuracy.
The Impact of Copyright Laws and Open Access
Copyright laws established in the 1920s restricted access to scholarly research until the early 2000s when open access became more prevalent. These policies will significantly influence AI generation and, consequently, how the past is imagined. Magnani believes that teaching students to approach GenAI cautiously will lead to a more technically literate and critical society.
Looking Ahead: The Future of AI in Archeology
This study is part of a series that Magnani and Clindaniel are conducting to explore the use of AI in archaeological research and topics. Their ongoing work aims to address the distance between scholarship and AI-generated content, providing a valuable template for other researchers to follow.
The findings of this study raise important questions about the reliability of AI-generated information and the need for careful consideration of its sources. As GenAI continues to evolve, it is crucial to ensure that it accurately represents the past, and this study is a significant step towards achieving that goal.
