Carnegie Mellon’s Pokerbot models asymmetric data
On Feb. 12–13, the Association for the Advancement of Artificial Intelligence held a meeting to declare that Carnegie Mellon University’s Pokerbot — Baby Tartanian 8 — had won first place at the Annual Computer Poker Competition in the total bankroll category and third place in the bankroll instant run-off category, out of 11 participating teams. A “Pokerbot” is exactly what it sounds like: a robot that plays poker. Heads-Up, No-Limit Texas Hold’em poker, to be specific.
Baby Tartanian 8 was designed by Noam Brown, a Ph.D. student in the School of Computer Science, alongside his adviser, Tuomas Sandholm, a professor in the Computer Science Department. The project is representative of an incomplete information problem. The aim of this problem is to be able to find and play the strategy with the best results, based off limited information.
“This algorithm is not specific to poker,” Brown said. “It should be generally applicable to any strategic interaction where you have multiple agents and asymmetric information.” Asymmetric information is a situation where one person has information that the other person doesn’t. Examples of this type of situation include negotiation situations, security situations, and transactions.
Tartanian 8 was developed as a successor to the previous bot Tartanian 7 using the Comet supercomputer at the San Diego Supercomputer Center. Tartanian 8 was built entirely from scratch, boasting better, faster algorithms, and hardware that supports increased computation power.
“The one thing that we used this year that we didn’t use in the previous years was the idea of pruning,” Brown said. “In a game like poker, where you have a lot of actions available to you, pruning helps cut down to actions that are worth investigating according to a given situation, instead of considering all the actions available to you.”
A paper published recently by Brown and Sandholm explains the idea of pruning as a way to minimize the options that would be investigated, while ensuring that the algorithm still arrives at an optimal strategy. However, due to the competition’s restrictions on the participants, Tartanian 8 had to be scaled down to Baby Tartanian 8, a 200 GB version with two cores.
Baby Tartanian 8 was developed after the “Brains vs. Artificial Intelligence” exhibition organized by Carnegie Mellon last year, where Carnegie Mellon invited top poker players to compete against Tartanian 7. Although three of the four human players had higher earnings than Tartanian 7, they had not won by enough of a margin to consider their win to be of statistical significance. This means that it wasn’t possible to distinguish with certainty about who was better.
After 80,000 hands of poker, played over a span of two weeks, the researchers, with feedback from the poker players, were able to identify the strengths and weaknesses of the bot. The strengths were that Tartanian 7 often did things that humans wouldn’t. For instance, it would bet a very large sum of money on a very small pot or a very tiny amount to a very large pot. As opponents, they said that such strategies threw them off. According to them, that was one instance when the bot was unpredictable, as human players generally don’t make such decision, but the bot was unfazed, and was able to balance that move with other moves.
The researchers also identified some of the weaknesses of the bot. One of the major weaknesses of the bot was that the algorithm was designed to put card hands into similar sets of hands on which it could use a similar strategy. This is called abstraction. When grouping those hands together, the bot would sometimes misjudge the situation, which would cause it to have it a suboptimal strategy and act absurdly.
This feedback was critical for the development of the newest bot, and future plans for the Tartanian model involve upgrading the algorithm and equipping it with better hardware that is more suited to perform higher computation levels.
“The ultimate goal in this line of research is to eventually beat the top humans,” Brown said. “Hopefully, in the next 2–3 years, we will be able to accomplish that.”