Berkeley robots learn to walk on their own in record time
October 26, 2022 by Marnie Ellery
Berkeley researchers may be one step closer to making robotic dogs our new best friends. Using advances in machine learning, two separate teams have developed sophisticated approaches to shortening in-field training times for quad robots, and making them walk — and even roll — in record time.
For the first time in robotics, a team led by Sergey Levin, associate professor of electrical engineering and computer science, has demonstrated a robot that learns to walk without prior training from models and simulations in just 20 minutes. The show is a major advance, as this robot has relied solely on trial and error in the field to master the movements needed to walk and adapt to different settings.
“Our work shows that training robots in the real world is more feasible than previously thought, and we hope, as a result, that other researchers will be able to begin tackling more real-world problems,” said Dr. Laura Smith. A student in Levine’s lab and one of the lead authors of Paper published on arXiv.
In previous studies, robots of comparable complexity required several hours to weeks of data entry to learn to walk using reinforcement learning (RL). Often, they were also trained in controlled laboratory settings, where they learned to walk over relatively simple terrain and received careful feedback on their performance.
Levine’s team was able to accelerate learning speed by taking advantage of advances in RL algorithms and machine learning frameworks. Their approach enables the robot to learn more efficiently from its mistakes while interacting with its environment.
Levin, Smith and Ilya Kostrkov, co-author of the research paper and a postdoctoral researcher in Levine’s lab, placed the robot in unstructured outdoor environments to demonstrate it. From a standing position, the robot takes its first steps, and after a few vibrations, it learns to walk constantly, as we have seen, for example, In this video of the robot on local fire paths.
According to Smith, the robot explores by moving its legs and observing how its actions affect its movement. “Using feedback on whether it is moving forward, the robot quickly learns how to coordinate the movements of its limbs in a normal walking gait,” she said. “While making progress up the hill, the robot can learn from its experience and hone its newfound skills.”
A different team at Berkeley, led by Peter Appel, a professor of electrical engineering and computer science, took another approach to helping a four-legged robot teach itself to walk. As shown in this videoThe robot starts on its back and then learns how to roll, stand and walk in just one hour of realistic training time.
The robot has also proven to be able to adapt. Within 10 minutes, I had learned to tolerate pushes or roll quickly and get back on my feet again.
Abel and his researchers used an RL algorithm called Dreamer that uses the acquired world model. This model is built using data collected from the robot’s ongoing interactions with the world. The researchers then trained the robot within this universal model, using the robot to imagine possible outcomes, a process they call “imagination training.”
In a paper published on arXivAbel and his team looked at how the Dreamer algorithm and global model could enable faster learning on physical robots in the real world, without simulations or demonstrations. Philip Wu, Alejandro Escontrella, and Danjar Haffner were co-authors of this paper, and Ken Goldberg, Professor of Industrial Engineering, Operations Research, Electrical Engineering, and Computer Science, was a co-author.
The robot can use this world model to predict the outcome of an action and decide what action to take based on its observations. This model will continue to improve and become more accurate as the robot explores the real world.
“The robot dreams and imagines what the consequences of its actions will be and then trains itself in the imagination to improve it, to think of different actions and to explore different sequences of events,” Escontrella said.
Additionally, the dog robot must figure out how to walk without any reset from the researchers. Even if the bot takes a bad fall in the exploration process or is pushed down and recovery seems impossible, it should still try to recover on its own without any intervention. “It’s up to the robot to learn,” Escontrella said.
These recent studies by both teams demonstrate how rapidly RL algorithms are evolving, with research teams at UC Berkeley driving innovation. last yearJitendra Malik, a professor of electrical engineering and computer science and a research scientist at the Facebook AI Research Group, first introduced an RL strategy called rapid motor adaptation (RMA). With RMA, bots learn behavior using a simulator and then rely on that learned behavior to generalize to real-world situations.
This approach was a significant improvement over existing learning systems, and the latest RL curriculum used by the Levine and Abbeel teams takes one step further by enabling bots to learn from their real-world experience.
“Any machine learning model will inevitably fail to generalize in some circumstances, usually when they differ significantly from their training conditions,” Smith said. “We are studying how to allow robots to learn from their mistakes and continue to improve as they work in the real world.”
Not relying on simulators also provides practical benefits, such as increased efficiency. “The main takeaway is that you can do reinforcement learning directly on real bots in a reasonable time frame,” Abel said.
Both research teams plan to continue teaching robots increasingly complex tasks. One day, we may see four-legged robots accompanying search and rescue teams or assisting with microsurgery.
“It’s worth noting how amazing all these different ways of learning are,” Escontrella said. “Ten to 20 years ago, people were manually designing controllers to determine leg movement. But now a robot can learn that behavior just from data and a simple reward signal — and continue learning into the future. In 10 years, we’ll say all that work has been small steps.” , Literally “.
#Step #step #Berkeley #Engineering