Disney conducts research with CMU

Carnegie Mellon University has welcomed Disney, the entertainment giant, to campus to fund a research laboratory under a five-year contract. The lab is situated close to the School of Computer Science complex, where robotics and animation technologies will be developed for Disney’s entertainment empire.

According to the Pittsburgh Post-Gazette, eight Disney scientists will team with CMU researchers and students to create ways of effective interaction between robots (or virtual characters) and theme park visitors.

Joe Marks, the vice president for research and development at Walt Disney Imagineering and Walt Disney Animation Studios, said, “We are looking forward to starting the collaboration, and hoping to contribute to the CMU community.”

Explaining how the collaboration can be beneficial for CMU as well, he said, “Collaborating with industry can make universities technically strong. It also makes the area have both strong universities and strong companies, and thus be more attractive to faculty candidates.”
According to Marks, Carnegie Mellon was chosen because of its strength in robotics, human-computer interaction, computer vision, and speech recognition.

Disney will also open a lab at the Swiss Federal Institute of Technology Zurich, also known as ETH Zurich. Marks hopes that the two labs will cooperate with each other and use their different fields of expertise to develop new technology.

Jessica Hodgins, a robotics and computer science professor at Carnegie Mellon, and also the director of Disney labs in Pittsburgh, shared Carnegie Mellon’s perspective on this joint initiative.
“We believe some work can be done better together with Disney than working separately,” Hodgins said.

According to Hodgins, the uniqueness of this collaboration is highlighted by the fact that historically, the entertainment industry has never been involved in academic research.
Explaining the reason for Disney’s current involvement with academia, Marks said, “We need to be more open to the rest of the world and be intertwined with the R&D communities in order to remain in the front.”

The launch of the co-labs with Carnegie Mellon and ETH is the first step Disney made toward their visionary collaboration with academia. Meanwhile, through this development, CMU’s leadership in the area of entertainment technologies becomes firmly established.
Hodgins said that the main task now is designing autonomous robotic characters that can entertain crowds at both Disney theme parks. She referred to Turtle-talk which is currently operating at Epcot Center at Walt Disney World in Florida. In Turtle-talk, Crush, of Finding Nemo fame, entertains visitors and talks to them.

An image of the turtle is projected on the screen and a human controller speaks for the turtle. Hodgins added that this system has not yet been perfected. “For example, currently, a person is needed to hold a microphone in order to let children talk to the turtle, which is not convenient and some children could be scared by the microphone,” she said.

In general, the research will mostly focus on developing software to manipulate the existing hardware. The long-term goal is to develop robots that can interact with people, and can interpret human behaviors and respond appropriately.

Hodgins mentioned that this project will benefit Carnegie Mellon students in a number of ways. Firstly, the lab will hire researchers at the Ph.D. level and provide research opportunities.

Secondly, Disney employees will be involved with the campus community — some of them could become adjunct faculty members and co-advise students. The Disney employees could also deliver lectures to share their unique experience and knowledge from the entertainment industry.
Hodgins emphasized the fact that Disney will be able to provide Carnegie Mellon with an important source of real-world problems and data sets to work on.

The efficacy of entertainment technologies is difficult to assess unless it is put into real scenarios and evaluated by the customers. On this, Hodgins commented, “We would install things in the theme park either for permanent use or for test setup over a certain period of time which later on will be removed.”

Marks claimed that Disney has many threads of research to explore and has a broad range of interest in the areas of computer vision, audio/speech recognition, and artificial intelligence.
Marks also wants to expand Disney’s interaction with Carnegie Mellon beyond just the Robotics and Human-Computer Interaction institutes.

“We have a family of business that can provide many research opportunities: radio entertainment, TV network, toy business, ESPN sport, theme park, and game business.”