The techniques developed to help DeepMind play the almost decade-old strategy game are now being used to quickly retrain neural networks for self-driving vehicles.
Finding new ways to train neural networks is becoming more important all the time, especially as the race to develop autonomous cars heats up. This has led developers to come up with some reasonably inventive ways of getting their networks up to speed, and one of them seems to involve the game StarCraft II.
What could a nearly decade-old game possibly have to do with training state-of-the-art neural networks? More than you’d think actually. See, according to a report published Thursday by MIT Technology Review, the techniques that some people are applying to make the game’s AI smarter and harder to beat can be carried over to neural network development.
See, in StarCraft II, you’re tasked with controlling dozens of individual units, each with unique skills, all while managing resources and fighting an opponent who is trying to wipe you out. It’s a complicated task that humans are good at, but machines can struggle with. Sound kind of familiar?
(Hint: it’s a lot like driving a car.)
What’s happening with StarCraft II’s AI is that Google’s DeepMind system is using a type of algorithm called population-based training to mimic natural selection. The algorithm shortcuts the learning process by starting with the most efficient units and then basing future adaptations on those.
The same thing is happening with self-driving development. DeepMind is selecting the aspects of the neural network that are most efficient and using those when it needs to retrain or adjust its procedures as new data comes in.
“One of the key challenges for anyone doing machine learning in an industrial system is to be able to rebuild the system to take advantage of new code,” Matthieu Devin, director of machine learning infrastructure at Waymo, said in an interview with MIT Technology Review. “We need to constantly retrain the net and rewrite our code. And when you retrain, you may need to tweak your parameters.”
Google has already commercialized some of its machine-learning technology, but it’s specifically using this population-based training model on its fleet of Waymo self-driving vehicles, which is already viewed by many as the most advanced self-driving car program in the world and one that has driven billions and billions of simulated miles.