Image: © AdriaVidal/Stock.adobe.com
Members of the scientific community were critical of the demo, with testers saying it produced biased content that looked real but was essentially ‘pseudo-science’.
Meta’s latest AI tool has been taken down after being available to the public for just three days.
Last week, the tech giant released a public demo of its science-focused Galactica AI that anyone could try. But the tech was met with intense criticism from users who said it generated biased and misleading content.
Galactica was designed to help with scientific tasks, being able to “store, combine and reason about scientific knowledge”, according to Meta. The company said the model could create its own scientific papers or Wikipedia-style articles, with relevant references and formulas included.
Galactica is a large language model, which is an AI system trained on a massive volume of text. This AI was trained 48m examples of scientific articles, websites, textbooks, lecture notes and encyclopaedias.
In a collaboration with Papers With Code, Meta AI made Galactica open-source and launched a website where people could test the model.
‘Dangerous’ development
The director of the Max Planck Institute of Intelligent Systems, Michael Black, was critical of Galactica when he tested it. On Twitter, Black said he was “troubled” after asking the AI about various topics he knew about.
“In all cases, it was wrong or biased but sounded right and authoritative,” Black said. “I think it’s dangerous.”
Black added that the content Galactica produced looked realistic but was essentially “pseudo-science”. He said the AI model risks creating “an era of deep scientific fakes”, as it could be difficult to detect AI-generated papers from others.
“I applaud the ambition of this project but caution everyone about the hype surrounding it,” Black said. “This is not a great accelerator for science or even a helpful tool for science writing.”
Why dangerous? Galactica generates text that's grammatical and feels real. This text will slip into real scientific submissions. It will be realistic but wrong or biased. It will be hard to detect. It will influence how people think. (5/9)
— Michael Black (@Michael_J_Black) November 17, 2022
A machine learning engineer who tested Galactica, Enric Domingo, was initially supportive of the AI. He said on Twitter that it was not “perfect” but was “a very interesting tool to search about new topics” and that it provided the sources of its results.
However, after more testing, Domingo said Galactica only provides references and sources when it generates “a wiki article” and he suspected these were not accurate or were missing most sources.
MIT Technology Review noted that there is research that highlights the limitations of large language models, but tech companies like Meta appear to have a blind eye to these issues.
The setback likely won’t be the end for Meta’s endeavours into large language models. In May, the company gave AI researchers access to one of its models that has 175bn parameters trained on publicly available datasets.
10 things you need to know direct to your inbox every weekday. Sign up for the Daily Brief, Silicon Republic’s digest of essential sci-tech news.
