Image: DeepMind
In 10 coding challenges, AlphaCode ranked within the top 54pc of human programmers.
DeepMind, Alphabet’s artificial intelligence subsidiary based in the UK, has created a new AI-powered system called AlphaCode that it says can write computer programs “at a competitive level”.
The company tested the system by simulating participating in 10 recent contents run by Codeforces, a website that hosts competitive programming competitions.
It found that AlphaCode ranked within the top 54pc of programmers who had taken part by solving problems that require a combination of critical thinking, natural language processing and human-like creativity.
DeepMind, which was acquired by Google in 2014, said that the competitions selected for evaluation were all newer than the data AlphaCode was trained on, and the system performed about as well as the median competitor.
Codeforces founder Mike Mirzayanov said that despite his scepticism, AlphaCode exceeded his expectations.
“I was sceptical because even in simple competitive problems it is often required not only to implement the algorithm, but also – and this is the most difficult part – to invent it. AlphaCode managed to perform at the level of a promising new competitor,” he said.
Problem-solving abilities such as the ones needed for Codeforces competitions are usually considered beyond the ability of existing AI models. However, DeepMind said that AlphaCode was able to stand its ground because of advances in large-scale transformer models combined with sampling and filtering.
Transformer models allow AI systems to write code much in the way Gmail’s Smart Compose feature predicts text when you’re writing an email. A transformer architecture is also used by Microsoft-owned Github’s predictive text for programmers tool, GPT-3.
Petr Mitrichev, a Google software engineer and competitive programmer, expressed his surprise at the performance of AlphaCode because competitive programming “is a really hard thing to do, requiring both good coding skills and problem-solving creativity in humans”.
“I was very impressed that AlphaCode could make progress in this area, and excited to see how the model uses its statement understanding to produce code and guide its random exploration to create solutions,” he said.
DeepMind has made many AI breakthroughs in recent years. In 2020, it claimed that its AlphaFold AI program solved a half-century-old puzzle of protein folding, and went on to establish a detailed database of proteins last year.
Don’t miss out on the knowledge you need to succeed. Sign up for the Daily Brief, Silicon Republic’s digest of need-to-know sci-tech news.
