Image: © Lemonsoup14/Stock.adobe.com
Meta’s self-taught evaluator is part of a new round of AI innovations developed by the company’s research division team, following its introduction in an August paper.
Social media platform Meta has announced the release of several new AI models, including the self-taught evaluator, which the company says can train other AI without the need for human input, in the hope that it will improve efficiency and scalability for enterprises using large language models (LLM).
The tool was first introduced in a paper released in August, where it was stated that the technology uses the same ‘chain of thought’ method previously seen in recent Open AI o1 models, to better generate reliable response and judgements.
With human involvement in LLMs, evaluation is often considered slow and more costly. Meta’s self-taught evaluator aims to create a training approach that eliminates the need for human-labelled data. The AI is prompted to break down difficult problems into manageable, practical steps, improving the accuracy of responses on a wide range of complex topics – for example science, coding and maths.
Meta’s research is part of a growing trend in which techniques incorporate LLMs as part of the automation and improvement process. Jason Weston, a Meta research scientist, told Reuters: “We hope, as AI becomes more and more super-human, that it will get better and better at checking its work, so that it will actually be better than the average human.
“The idea of being self-taught and able to self-evaluate is basically crucial to the idea of getting to this sort of super-human level of AI,” he said.
However, concerns remain about the risks around AI tools, especially those that have less human involvement. Earlier this year, the team at William Fry discussed the importance of humans in the loop when it comes to AI.
When it comes to Meta specifically, the company is no stranger to criticism over its AI tools. In 2022, the its science-focused Galactica AI tool had to be pulled down after just three days after users said it generated biased and misleading content.
And even as the AI models advance, company claims about AI tools can be difficult to verify. A report from the AI Index earlier this year claimed robust evaluations for LLMs are “seriously lacking” and there is a lack standardisation in responsible AI reporting.
Don’t miss out on the knowledge you need to succeed. Sign up for the Daily Brief, Silicon Republic’s digest of need-to-know sci-tech news.
