Reviewed by Alex SmithApr 7 2022
Recently, the Association for Computing Machinery (ACM) has bestowed the 2021 ACM Prize in Computing to Pieter Abbeel for his contributions to robot learning, including deep reinforcement learning for robotic control and learning from demonstrations.
Image Credit: Shutterstock.com/PopTika
Abbeel initiated programming robots to learn from human demonstrations (“apprenticeship learning”) as well as through their own trial and error (“reinforcement learning”), which has shaped the basis for advanced robotics. Abbeel is the Co-Founder, President and Chief Scientist at Covariant – an AI robotics company, and a Professor at the University of California, Berkeley.
The ACM Prize in Computing distinguishes early-to-mid-career computer scientists whose research efforts have had a major impact and extensive implications. The award comprises a cash prize of $250,000, from an endowment offered by Infosys Ltd.
At the beginning of his career, Abbeel formulated new apprenticeship learning methods to considerably enhance robotic manipulation. As the domain developed, scientists could teach robots to observe and operate rigid objects such as spoons or wooden blocks.
However, teaching robots to operate deformable objects, like cloth, proved tough as the way soft materials shift when touched is erratic. Abbeel contributed new approaches to improve physics-based monitoring, robot visual perception, control and learning from demonstration.
By integrating these new techniques, Abbeel created a robot that could fold clothes such as shirts and towels — an upgrade over prevailing technology that was recognized as a critical breakthrough at the time.
Abbeel’s contributions also include building robots that could conduct surgical suturing and spot objects and map their trajectories in undefined situations.
In recent times, he has established “few-shot imitation learning,” where a robot could learn to carry out a task from merely one demonstration after having been pre-taught with several demonstrations on connected tasks.
Another particularly favorable area where Abbeel has delivered critical contributions is deep reinforcement learning for robotics. Reinforcement learning is a domain of machine learning where an agent (e.g., a computer program) aims to move toward a reward (e.g., winning a game).
Although early reinforcement learning programs were operational, they could only carry out simple operations. The innovation of merging reinforcement learning with deep neural networks paved the way for the new domain of deep reinforcement learning, which can solve a lot more intricate issues than computer programs designed with just reinforcement learning alone.
Abbeel’s major innovative contribution in this domain was creating a deep reinforcement learning technique called Trust Region Policy Optimization. This technique steadies the reinforcement learning process, allowing robots to learn a variety of simulated control skills.
By distributing his outcomes, uploading video tutorials and reporting open-source code from his lab, Abbeel helped bring together a community of scientists that has since navigated deep learning for robotics to new levels further — with robots carrying out ever more complex operations.
Abbeel has also delivered numerous other original contributions, including soft-actor critic, which is one of the most in-demand deep reinforcement learning algorithms up to now; hindsight experience replay, which has been influential for deep reinforcement learning in sparse-reward/goal-motivated environments; generalized advantage estimation, which supported the first 3D robot locomotion learning; domain randomization, which emphasizes how learning across suitably randomized simulators can generalize amazingly well to the real world.
Teaching robots to learn could spur major advances across many industries—from surgery and manufacturing to shipping and automated driving. Pieter Abbeel is a recognized leader among a new generation of researchers who are harnessing the latest machine learning techniques to revolutionize this field.
Gabriele Kotsis, President, ACM
Abbeel has made leapfrog research contributions, while also generously sharing his knowledge to build a community of colleagues working to take robots to an exciting new level of ability. His work exemplifies the intent of the ACM Prize in Computing to recognize outstanding work with ‘depth, impact, and broad implications,’” Gabriele Kotsis added.
Infosys is proud of our longstanding collaboration with ACM, and we are honored to recognize Pieter Abbeel for the 2021 ACM Prize in Computing. The robotics field is poised for even greater advances, as innovative new ways are emerging to combine robotics with AI, and we believe researchers like Abbeel will be instrumental in creating the next great advances in this field.
Salil Parekh, Chief Executive Officer, Infosys
Abbeel will be officially given the ACM Prize in Computing at the yearly ACM Awards Banquet, which will be conducted on June 11th at the Palace Hotel in San Francisco.