Humans are considered social creatures and learn from each other, even from a young age. Infants keenly notice their siblings, parents, or caregivers. They watch, imitate, and then replay what they see to learn skills and behaviors.
SCS and Meta AI researchers have announced RoboAgent, an AI agent that leverages passive observations and active learning to enable a robot to acquire manipulation abilities on par with a toddler. Image Credit: Carnegie Mellon University.
The approach in which babies learn and explore their surroundings inspired scientists at Carnegie Mellon University and Meta AI to come up with a new way to teach robots how to concurrently learn multiple skills and leverage them to handle unseen, daily tasks.
The scientists set out to come up with a robotic AI agent with manipulation abilities that is comparable to a 3-year-old child.
The research group has declared RoboAgent, an artificial intelligence agent that leverages active learning and passive observations to allow a robot to obtain manipulation abilities on par with a toddler.
RoboAgent is a critical milestone toward general robotic agents that are efficient learners, effective in novel situations and capable of expanding their behaviors over time.
Vikash Kumar, Adjunct Faculty, School of Computer Science’s Robotics Institute, Carnegie Mellon University
Kumar added, “Current robots are highly specialized and trained for individual tasks in isolation. In contrast, we set out to create a single artificial intelligence agent capable of exhibiting a wide range of skills in unseen scenarios. RoboAgent learns like human babies — leveraging a combination of abundant passive observations and limited active play.”
RoboAgent has the potential to finish 12 manipulation skills throughout varying scenes. This study performed refers to a robotic learning platform adaptable to altering environments. Dissimilar to the past research, the team illustrated their work in real environments—not simulation—and did so with far fewer data compared to previous projects.
RoboAgents are capable of much richer complexity of skills than what others have achieved. shown a greater diversity of skills than anything ever achieved by a single real-world robotic agent with efficiency and a scale of generalization to unseen scenarios that is unique.
Abhinav Gupta, Associate Professor, Robotics Institute, Carnegie Mellon University
The team’s agent learns via a combination of passive observations and self-experiences contained in internet data. Just like how a parent would guide their child, scientists teleoperated the robot via tasks to offer it beneficial self-experiences.
“The effectiveness and efficiency of our approach stem from our novel policy architecture that allows our agents to reason even with limited experiences. RoboAgent acts in response to specified text/visual goals by predicting and aggregating decisions in terms of temporal chunks of movements instead of commonly used per-timestep actions,” stated Homanga Bharadwaj, a Ph.D. student in robotics.
Robots mainly learn from their experiences, not from what occurs passively around them. This innate blindness to what goes on in their environment fundamentally restricts the diversity of experiences robots are exposed to and their potential to adapt to new situations.
To such limitations to be defeated, RoboAgent learns from videos on the internet—dissimilar to how babies acquire behaviors and knowledge by noting their surroundings passively.
“RoboAgent leverages the information contained in these videos to learn priors about how humans interact with objects and use various skills to successfully complete tasks. Additionally, observing similar skills in multiple scenarios allows it to learn what is and isn't necessary to complete a task. It leverages these lessons when presented with unknown tasks or unseen environments,” stated Mohit Sharma, a Ph.D. student in robotics.
An agent capable of this sort of learning moves us closer to a general robot that can complete a variety of tasks in diverse unseen settings and continually evolve as it gathers more experiences.
Shubham Tulsiani, Assistant Professor, Robotics Institute, Carnegie Mellon University
Tulsiani added, “RoboAgent can quickly train a robot using limited in-domain data while relying primarily on abundantly available free data from the internet to learn a variety of tasks. This could make robots more useful in unstructured settings like homes, hospitals, and other public spaces.”
The team is open-sourcing its codebase, trained models, hardware drivers, and—most remarkably—the complete data set gathered in this research. RoboSet is known to be the biggest publicly available robotics data set on commodity hardware.
The research group believes that this will allow others to reuse, adapt, and pass it forward, resulting in a truly foundational general robotic agent concerning time.