Challenging human intelligence, Alphabet (Google’s parent company) subsidiary DeepMind has created an artificial intelligence system called AlphaCode that “writes computer programs at a competitive level.”
This marks the first time an AI code generation system has reached a competitive level of performance in programming competitions.
AlphaCode achieved an estimated rank within the top 54 per cent of participants in programming competitions by solving new problems that require a combination of critical thinking, logic, algorithms, coding, and natural language understanding,” the company said in a statement late on Wednesday.
AlphaCode uses transformer-based language models to generate code at an unprecedented scale, and then smartly filters to a small set of promising programs.
Creating solutions to unforeseen problems is second nature in human intelligence, a result of critical thinking informed by experience.
The machine learning community has made tremendous progress in generating and understanding textual data, but advances in problem-solving remain limited to relatively simple maths and programming problems, or else retrieving and copying existing solutions.
The company validated AlphaCode’s performance using competitions hosted on Codeforces, a popular platform that hosts regular competitions that attract tens of thousands of participants from around the world who come to test their coding skills.
“We selected for evaluation 10 recent contests, each newer than our training data. AlphaCode placed at about the level of the median competitor,” said DeepMind.
DeepMind was acquired by Google in 2014. The company is based in London, with research centers in Canada, France, and the US.
“Overall, AlphaCode placed at approximately the level of the median competitor. Although far from winning competitions, this result represents a substantial leap in AI problem-solving capabilities and we hope that our results will inspire the competitive programming community,” the company noted.
Mike Mirzayanov, Founder of Codeforces, said the results of AlphaCode exceeded his expectations.
“I was skeptical because even in simple competitive problems it is often required not only to implement the algorithm but also (and this is the most difficult part) to invent it. AlphaCode managed to perform at the level of a promising new competitor. I can’t wait to see what lies ahead,” he elaborated.
DeepMind said it is releasing the dataset of competitive programming problems and solutions on the open-source code repository service GitHub.
