Artificial intelligence (AI) has already mastered trouncing humans at games like Chess and Go. But a new DeepMind algorithm now seems capable of shelving its competitive streak for something far more challenging: Cooperation.
According to a study published today in the journal Science, DeepMind researchers have designed automated “agents” to team up with machines and humans alike to play a version of the video game Quake III Arena modeled on capture the flag.
In the arena of artificial intelligence, this constitutes a major breakthrough. Never before has a computer been proficient enough at complex social interactions—like predicting how others will behave—to navigate and win at multiplayer games. DeepMind has dubbed the triumph a “human-level performance.” But of course, even the most complex games remain a far cry from real life.
In the game, each of two groups of players tries to capture as many of the other team’s flags as possible. After starting out at base camps at opposite ends of a map, players dart around an intricate maze of obstacles, attempting to steal the other side’s flag and bring it back to their side of the map. If a player is tagged by an opponent, they’re forced back to base to “respawn” and begin the hunt from scratch.
Over the course of 450,000 games—comparable to four years of game play—pairs of AI agents taught themselves to maneuver through the maze while tagging and evading opponents. The AIs had access only to the same rules a human player would, and each learned independently from its own experience. At first, the agents made choices at random, but through a brute force trial and error technique, each began to file away tips and tricks that would increase its rate of success.
The researchers then tested their freshly trained AI agents in a tournament that included 40 human players. In some cases, people were matched with AI partners; in others, they tried to beat them. But none of the human participants knew whether their teammates or opponents were human or machine.
Unlike people, however, AIs are able to process information and reaction almost instantaneously during gameplay—so DeepMind incorporated a 267-millisecond delay into the program. But this tweak didn’t fully level the playing field.
On average, AI-AI and human-AI duos captured more flags than human-human pairs. And when human players reported which teammates they preferred, the majority of them were machines.
However, in a separate test, two professional game testers were able to best AI agents after 6 hours of training on an advanced map, Philip E. Ross reports for IEEE Spectrum.
Since wrapping up the Quake III Arena project, DeepMind’s engineers have already designed a second system that has now mastered multiplayer settings on Starcraft II, a strategy game set in space. But the team has their sights set on far more. Eventually, they plan to introduce their AIs to challenges that dwarf wins in the world of gaming—like self-driving cars or robot-guided surgeries.
“Games have always been a benchmark for AI,” OpenAI’s Greg Brockman, who was not involved in the study, told Cade Metz at The New York Times. “If you can’t solve games, you can’t expect to solve anything else.”
But AI still has a lot to learn, and it’ll be a long time before they’re ready for more advanced (and higher stakes) applications, Jaderberg told Edd Gent at Science.
Even the agents racking up points in Quake III Arena aren’t really “collaborating,” Mark Riedl, an artificial intelligence expert at Georgia Tech College of Computing who was not involved in the study, told Metz. Each responds individually to the conditions it observes in the game, while human players can check in with each other over messages or through actual conversation.
On a real battlefield, AI would still be stumped—and that’s unlikely to change anytime soon.
