The matchup started with a bang. After five tossup questions, the score was Computer 65, Humanity 0. In a video of the match, one of the human players can be seen sticking his tongue out at the robot.
While the match between assistant professor Jordan Boyd-Graber’s quiz-bowl-playing robot, QANTA, and four human Jeopardy champions eventually ended in a tie, Boyd-Graber was pleased overall with its performance. QANTA faced off against its human competitors on stage at the National Academic Quiz Tournaments’ High School National Championship Tournament in late May in Chicago.
“One of the champions said it was like a very good eighth-grader,” he said, adding that QANTA was very good at questions that involved people, places and things, while it struggled with more abstract concepts, pop culture and current events.
While the idea of a question-answering robot isn’t new – IBM’s “Watson” computer faced off against Jeopardy winners Ken Jennings and Brad Rutter in 2011 – QANTA includes some novel machine-learning functionality. For instance, while Jeopardy contestants have to wait until the question is completed before buzzing in, quiz bowl contestants can buzz in as soon as they think they have enough information to answer the question correctly.
Therefore, Boyd-Graber explained, there are two steps to QANTA’s process. The first is generating a big pool of guesses based on the information that has been loaded into the computer from sources like Wikipedia and more than 20 years of quiz bowl questions. The second step is deciding when it has enough information to identify the best guess.
“The process of figuring out a guess is where the scientific novelty comes in,” Boyd-Graber said. Check out more on the technology behind QANTA in this introductory video or on graduate student Mohit Iyyer's website.
In addition to putting the robot through its paces, Boyd-Graber was using the high school tournament match as an outreach effort for computer science. Because the project is open source, he is hoping that students will want to contribute and get interested in computer science in the process. In fact, Boyd-Graber will be recruiting undergraduate students this fall to create an interface that will allow users to interact with the robot without having to know programming, and he encourages interested students to contact him.
QANTA will need those contributions as it prepares for its next challenge – taking on Ken Jennings at the University of Washington in Seattle on Oct. 2.
In preparation for the match, Boyd-Graber and collaborator Mohit Iyyer from the University of Maryland are loading up QANTA with even more information. While the robot has already “read” all of the Wikipedia pages that are answers to past quiz bowl questions, they’ll now be adding all pages linked to those answers. They will also be improving QANTA’s ability to understand art by identifying what objects can be found in what paintings, as well as scaling up QANTA to run on clusters rather than one laptop.
QANTA’s underlying technology ties back to Boyd-Graber’s main research interest, which is studying how humans interact with artificial intelligence as machine learning becomes more and more common in our everyday lives.
“We need to understand it, and it needs to understand us,” he said.