Researchers from the Moscow Institute of Physics and Technology have taken the second place in the Learn to Move — Walk Around challenge held in the runup to the 33rd Conference on Neural Information Processing Systems. The contest’s organizer, Stanford Neuromuscular Biomechanics Laboratory tasked the participants with creating a physiologically plausible 3D human skeleton model that walks around following velocity commands.
The MIPT team comprised two Neural Networks and Deep Learning Lab researchers: Sergey Kolesnikov and Valentin Khrulkov, also a doctoral student at Skoltech. The competition featured 70 teams, 15 of which made it into the final round. The MIPT duo was second only to a Chinese team from the NLP department of Baidu.
The contestants worked with a virtual human skeleton in the OpenSim simulator. Their goal was to train the skeleton to move in a particular direction at a given speed. Both of the values changed with time and were fed to the model in the form of a vector field. The winners were determined based on a quality metric that reflects how closely the skeleton’s movement follows the speed and direction required.
The organizers had the contest participants use an approach known as reinforcement learning (RL), and the MIPT duo employed Catalyst, an open-source framework with high-level utilities for deep learning and reinforcement learning research.
Kolesnikov noted that contests of this kind are yet another format for exchanging and accumulating experience, which the scientific community is increasingly embracing in addition to the traditional conferences, workshops, and publications.
“Such contests serve as a platform for testing lots and lots of entirely new ideas and hypotheses in a rapid and competitive manner,” the researcher went on. “In the first round, the participants have a fairly large amount of time to explore the environment and the problem at hand. In the second round, there is just a week or two to solve a highly specific final task. This teaches one to work both in the long-term and in the short-term mode, with the former promoting a better understanding of the problem and biomechanics, and the latter emphasizing the result.”
For Kolesnikov, this is not the first success at a NeurIPS RL contest. In 2017, the researcher teamed up with Mikhail Pavlov, earning a third-place finish behind the competitors from China and a Jürgen Schmidhuber team. In 2018, Kolesnikov placed third again with his teammates Alexey Grinchik (Skoltech) and Anton Pechenko, while the top two prizes went to the Baidu and Schmidhuber teams, with slightly higher scores. This year the MIPT team has moved one step closer to a first-place finish, which we hope they can land at NeurIPS RL 2020!
Held since 1989, NeurIPS is rated as an A* — top-grade — event by the Computing Research and Education Association of Australasia, which grades major international conferences in the field of computer sciences. NeurIPS has seen numerous breakthrough papers in the field of machine learning published in the recent years.