SciTech

Robot arm learns physical skills by playing Jenga

A robot designed by Carnegie Mellon alumnus Alberto Rodriguez and his team slowly pokes a wooden Jenga block out of a tower. Unlike traditional machine learning schemes, this robot takes careful visual and tactile measurements to model the behavior of Jenga blocks continuously, which has broad applications in manufacturing. (credit: Courtesy of the researchers via Massachusetts Institute of Technology) A robot designed by Carnegie Mellon alumnus Alberto Rodriguez and his team slowly pokes a wooden Jenga block out of a tower. Unlike traditional machine learning schemes, this robot takes careful visual and tactile measurements to model the behavior of Jenga blocks continuously, which has broad applications in manufacturing. (credit: Courtesy of the researchers via Massachusetts Institute of Technology)

For decades, engineers and computer programmers have been teaching robots how to play popular games like chess and Go. Artificial intelligence has even claimed the global top spot in some of these games, like when Google DeepMind’s AlphaGo software defeated Go world champion Lee Sedol nearly three years ago. Now, researchers at the Massachusetts Institute of Technology are training a robot to play Jenga.

Swahili for “to build,” Jenga was created by Leslie Scott in the early 1970s, based on a game played by children in her family. After stacking 54 rectangular wooden blocks into 18 layers, the objective of the game is to carefully remove blocks and place them at the top of the tower without toppling the entire structure.

The robot at MIT uses the industry-standard 6-axis RB 120 robotic arm from ABB (also used in electronic and food manufacturing) to interact with Jenga blocks. While it is certainly impressive to watch the robot prod at the tower and slowly push blocks out of their place, the real advancement is in the technique used to train the machine.

Traditional machine learning algorithms require extensive training data, covering all possible scenarios that might occur given a particular action. It would take tens of thousands of attempts at removing a block from a tower to gather enough data about Jenga. Additionally, since Jenga is a physical game that does not translate well to simulation, the robot needed training on how to operate in the real world.

Alberto Rodriguez leads the team that developed the Jenga robot. He is a Walter Henry Gale Career Development Assistant Professor in the Department of Mechanical Engineering at MIT and received his Ph.D. in robotics from Carnegie Mellon University. “Unlike in more purely cognitive tasks or games such as chess or Go, playing the game of Jenga also requires mastery of physical skills such as probing, pushing, pulling, placing, and aligning pieces,” Rodriguez said in an MIT press release. “It requires interactive perception and manipulation, where you have to go and touch the tower to learn how and when to move blocks. This is very difficult to simulate, so the robot has to learn in the real world, by interacting with the real Jenga tower. The key challenge is to learn from a relatively small number of experiments by exploiting common sense about objects and physics.”

The team at MIT devised a novel method of training their Jenga robot that relies on precise measurements and learning along the way. Initially, the robot arm randomly chose blocks in the tower and a location to push against. However, rather than extracting each block completely, it applied just enough pressure to get the relevant visual and force readings, and groups similar attempts into clusters representing block behaviors. After just 300 attempts, the robot had a reasonable idea about which blocks might be hard to move, easy to move, or topple the tower. Researchers even developed a model that allowed the robot to predict a block’s behavior given its current physical state based on visual and tactile measurements of the tower.

MIT graduate student Nima Fazeli, lead author of a paper in Science Robotics about the robot, says the clustering technique drastically improved the efficiency of the robot’s training phase and is motivated by human behavior. “The robot builds clusters and then learns models for each of these clusters, instead of learning a model that captures absolutely everything that could happen,” Fazeli explained.

Of course, the researchers couldn’t resist putting their robot to the test against human Jenga players and earlier machine learning algorithms. They used a simulation called MuJoCo to compare performance and found that even with the data collected through their sensors, other algorithms generated less accurate models than Rodriguez’s team. Additionally, while humans usually maintained an advantage against the robot, the gap between them was surprisingly small. Miquel Oller, another graduate student on the project, thinks the difference is due to the strategies used by experienced human contestants to plan their moves so that it becomes difficult for their opponents to extract blocks from the tower. For now, though, the researchers aren’t worried about rising to the Jenga throne.

The tactile learning system they developed could have massive applications beyond games with wooden blocks. According to Rodriguez, “There are many tasks that we do with our hands where the feeling of doing it ‘the right way’ comes in the language of forces and tactile cues. For tasks like these, a similar approach to ours could figure it out.”

Machines used on assembly lines today are hindered by their inability to respond to non-visual cues, and existing methods of teaching them physical skills require too much time and labor to be feasible. But this research could be applied to all tasks requiring physical interaction, including categorizing recyclable items and garbage as well as putting together consumer goods.

“In a cell phone assembly line, in almost every single step, the feeling of a snap-fit, or a threaded screw, is coming from force and touch rather than vision,” Rodriguez said to MIT News. “Learning models for those actions is prime real estate for this kind of technology."