Researchers Create Computer Program You’ll Never Beat At Poker

David Olson took home $303,909 for winning the 2014 World Series of Poker limit hold’em championship, a feat that simply wouldn’t have happened if a computer program named Cepheus had a say in the matter when it came down to heads-up play.

Researchers from the University of Alberta published a study in Science last Friday claiming to have created a computer program that is essentially unbeatable in heads-up limit hold’em.

Named after the King of Aetheiopia in Greek mythology who left Andromeda chained to a rock to be devoured, Cepheus was trained for 70 days playing matches against itself while considering 6 billion hands per second.

The program started out playing randomly and learned from every loss it suffered, called a “regret value.” The computer would store that regret value for subsequent hands and as matches added up the computer updated its strategy until it approached perfection.

Michael Bowling, the co-author of the study, said, “We’re not quite perfect, but we’re so close that even after a lifetime of playing against it, you wouldn’t know it wasn’t perfect.”

While epic matches in games like checkers and chess have been waged in the past between computers and humans, they differ from poker because it is a game with imperfect information—you only know your own hand and the cards on the table. When IBM’s Deep Blue beat Gary Kasparov in a chess rematch in 1997, the programmers had the advantage of having all of the game’s information displayed on the board.

In the process of programming Cepheus, Bowling and his colleagues were able to confirm some commonly held best practices for heads-up play. For example, the dealer in any given hand has an advantage of .088 big blinds per hand. They also confirmed it’s prudent to raise rather than call in the first move in the vast majority of hands.

Poker geeks can try their hand against Cepheus online. Freelance poker writer Christopher Hall did, and he claims to have come out slightly ahead against the program after 400 hands. While that’s a nice feat for Hall, a creation like Cepheus calls for an epic game against a pro like Phil Hellmuth to show that these researchers have truly solved the game.

(Header image via flickr)

 



Print This Post

Michael Tunney is a managing editor at Contently. He has also worked on marketing campaigns for bestselling authors like Robert Greene, Ryan Holiday, and James Altucher. Follow him on Twitter @mike_tunney.


Sort by:   newest | oldest | most voted
Jason B
Guest
Jason B

Could one not introduce elements of game theory to randomize decision making (particularly from the button with the positional advantage)? Or, if it’s learning from all of the past decisions that have come before, just start doing the opposite of what conventional gameplay would suggest?

(Heck I guess I could just go try these things out myself.)

D.Nelson
Guest
D.Nelson

It’s a limit game, so in theory you can develop a strategy for every possible hand to maximize your potential winnings regardless of what the opponent has. In that sense, it is just like perfect strategy for blackjack. holdem just has many more permutations of possible hands, and the computer could use brute force to learn how to maximize it’s chances of winning. I believe this was designed to play exclusively heads up too. I’d be much more curious about how the program would fare in a real poker game, with multiple players.

Steve
Guest
Steve

So it is essentially unbeatable, but the one person sited came out ahead? Not very strong support to the statement. Also, Phil Hellmuth is a very weird choice. Is that just the one poker player you’ve heard of? His specialty is no limit tournaments.

JamesDaBear
Guest
JamesDaBear

400 hands is a ridiculously-small sample size. I have no idea why the writer even included it.

Hellmuth has 3 WSOP bracelets in Limit Holdem and another in Limit Razz. 20 of his WSOP Final Tables are in games other than No Limit Holdem. I don’t think he’s the foremost authority on Limit Holdem, but he’d be a fine combination of skill, experience, notoriety and pride to go up against a computer.

Jeremy T
Guest
Jeremy T

I thought one of the major skills needed to play poker at a high level was the ability to read people. If you want a test against a top-level human player, it should really be a player who excels in online poker.

wpDiscuz