SciTech

Carnegie Mellon and IBM project ‘Watson’ competes on Jeopardy!

Credit: Courtesy of Wikimedia Commons Credit: Courtesy of Wikimedia Commons

“I am putting myself to the fullest possible use, which is all I think that any conscious entity can ever hope to do,” the HAL 9000 computer said aboard the Discovery One spacecraft in Stanley Kubrick’s science-fiction film, 2001: A Space Odyssey. The 1968 film — and the novel on which it was based — heavily featured interactions between humans and computers, with HAL showing as much intelligence as humans, and sometimes more. For years, scientists and engineers have pondered the possibility of computers possessing the same level of intelligence as humans, and recently, IBM teamed up with some researchers at Carnegie Mellon to create a computer program that may be on its way to making that possibility a reality.

The program, named “Watson,” is designed to interpret and answer questions written in natural human language. Eric Nyberg, a professor in the Language Technologies Institute who has been working with IBM to develop Watson, explained that while most consumer desktop computers have two or four central processing units (CPUs), Watson has over 2,000 powering it to answer posed questions in a matter of seconds. This is much faster than previous programs designed to excel at question answering (QA), which in the past could answer a question in just under a minute.

According to a commentary by Nyberg on www.pbs.org, “The big difference between a QA system like Watson and a search engine like Google is that Watson can read the text for you and provide a precise answer. Google will just give you a list of documents it thinks might contain the answer; you have to do the reading and answer-spotting yourself.”
Nico Schlaefer, a Ph.D. student in the Language Technologies Institute, also contributed to the development of Watson. In an interview on the Carnegie Mellon website, Schlaefer explained, “One tough challenge is that it’s an open-domain question answering program, which means that the questions can be pretty much about anything. So this requires the system to store all sorts of information that may be possibly relevant and process this information in real time.”

As a big test of Watson’s capabilities, the program competed on the game show Jeopardy! on Feb. 14–16 against greats Brad Rutter, who holds the all-time record for money won, and Ken Jennings, the record holder for the most consecutive wins. This was one of the most publicized human vs. computer contests since an IBM computer defeated world champion Garry Kasparov in a game of chess. Competing on Jeopardy! was going to be a challenge for Watson because of the style of language used within the game.

“The Jeopardy! language is very tricky — it’s very creative. It plays on all of our human abilities to understand language and all of the different forms that are used,” Nyberg explained.
The Jeopardy! challenge consisted of two matches, with the overall winner chosen based on the highest earnings over both games. Watson won the first match with $35,734, against Rutter’s $10,400 and Jennings’ $4,800. However, Watson had a few quirks. For example, there was one instance where Jennings posed an incorrect answer, and Watson guessed the same answer, reworded. Watson also showed some inconsistencies with the first game’s Final Jeopardy! question. Under the category of “U.S. Cities,” the question was, “Its largest airport is named for a World War II hero, and its second largest is named for a World War II battle.” Watson responded incorrectly with, “What is Toronto??????” The multiple question marks were indicative of Watson’s uncertainty of the answer.

A viewing party was held in the Rashid Auditorium in the Hillman Center for the second match on Feb. 16. Faculty and students alike piled into the standing-room-only auditorium to view a replay of the first match followed by the broadcast of the second match. In addition to sharing the historic moment with the audience, Nyberg and IBM executives spoke briefly about Watson’s technology and the history leading up to Watson’s debut on Jeopardy!.

The second match was slightly closer. Watson missed a Daily Double question for the first time and was in second after the first round as Jennings and Rutter were temporarily successful in consistently answering correctly before Watson. However, the match ended with Watson in the lead, and the total summed winnings over both matches were $77,147 for Watson, $24,000 for Jennings, and $21,600 for Rutter, which corresponded to prizes of $1 million, $300,000, and $200,000 for each of the players, respectively. Jennings and Rutter agreed to donate 50 percent of their winnings to charity, and IBM agreed to donate all of Watson’s winnings to charity.

Many see the success of Watson on Jeopardy! as a milestone in computing, though many computer scientists believe that computers may never possess the human traits of creativity or ingenuity. In a recent interview with the Associated Press, CIT Dean Pradeep Khosla commented, “The way to think about this is: Can Watson decide to create Watson...? We are far from there. Our ability to create is what allows us to discover and create new knowledge and technology.”