Poker Driving Artificial Intelligence Research 212
J-Hawker writes "The Canadian Press has a story about a University of Alberta team that is using Texas Hold-'em to study artificial intelligence. Poker seems to be a much more useful game for this research than chess. From the article: 'Poker has what are currently some of the biggest challenges to (artificial intelligence) systems, and uncertainty is the primary hurdle that we're facing,' said Michael Bowling, adding that the University of Alberta program was able to use its opponents' actions to infer certain things about their hands. 'The same techniques, the same principles that we're developing to build poker systems are the same principles that can be applied to many other problems. The nice thing about chess as a property of the game is what we call perfect information. You look at the board, you know where all the pieces are, you know whose turn it is — you have complete knowledge of the game,' he said. 'But in the real world, knowing everything is just so rare. Everything we do all day long is all about partial information. So poker's much more representative of what the real world's like, and in that sense it becomes a much harder problem.'"
Re:Bluffing (Score:5, Informative)
Re:Bluffing (Score:5, Informative)
Sudoku is trivial for an AI (Score:3, Informative)
Re:Straight Forward Evaluation (Score:5, Informative)
What confuses me is how the poker openings differ. I would speculate that a program would be some heuristic relating the ratio of bluffing to "playing the odds." I have gambling friends that play poker all the time and they have these rules that they follow when they play initially against people. They say it's the best until you "know" the people you're playing. Once you can read them then you deviate from the rules. The real irony is that the most successful people I know adhere to a system until they learn someone's movements. Sounds to me like I would write an application that specializes in playing the odds until it recognizes a historical action that statistically reveals the player is bluffing/not bluffing.
You can tell you don't play much poker.
Part of what differentiates a pro player to an amatur player in poker, is the ability to "project an image". A pro player will purposefully *project* an image of a bluffer, or a tight player, so that they can exploit that image of themselves when they see fit in the game.
Thusly, it is very difficlt to get a "read" on a good poker player, because not only do you not know what cards they have, but you don't know how they would play for any two given cards, so you can't use their behaviour to prdict the cards they have.
In the end, the above description is what any decent player is aiming for while they play.
Because of this, a computer can have a hard time going beyond implied odds calulations in determining how to play a hand - and any pro ill tell you, implied odds are a good starting point, but they won't make you money in the long run.
Re:If you are really interested... (Score:2, Informative)
Not true at all (Score:3, Informative)
Two players colluding in a game is a huge problem. Even a marginal advantage equals to a huge advantage played out over time.
Re:More info than a real player? (Score:3, Informative)
The effect of luck in poker win rates can still be seen over even 100000 hands. Google for "poker" and "confidence interval" for some in depth discussions on it.
Some Actual Factual Information (Score:1, Informative)
The Alberta folks are the world's leading (publicly known) authorities on poker AI. (There may be others who are using their knowledge to cheat at online poker, but they're not talking).
Here is the link to the Alberta web site:
http://www.cs.ualberta.ca/~games/poker/ [ualberta.ca]
(Note: you have scroll down a ways to the doc links).
If you want to read just one paper about computer poker, read "The Challenge of Poker". Here is a link (PDF warning):
http://www.cs.ualberta.ca/~darse/Papers/AIJ02.pdf [ualberta.ca]
Some of the Alberta guys spun off into a local company that makes an excellent software product, Poker Academy Pro:
http://www.poker-academy.com/poker-software/ [poker-academy.com]
This software written in Java and offers a programming API so you can plug your own bots into their game. The programming API has an online forum:
http://www.poker-academy.com/forums/viewforum.php
Here is the resources page of one of Alberta guys, now a senior developer working on Poker Academy:
http://spaz.ca/poker/ [spaz.ca]
And here is a blog written by a guy who has some excellent LGPL code available (left side of page);
http://pokerforprogrammers.blogspot.com/ [blogspot.com]
Note: do NOT use the version of this software on the Code Project - it is out of date and buggy - get the download from this blog.
There's lots of other stuff of variable quality...all the pages linked here are high-quality content in the sense that the authors have thought deeply about the AI problem and/or are very skilled programmers.
Enjoy,
Jeff
Re:Straight Forward Evaluation (Score:3, Informative)
A large part of what makes Hold'Em a unique challenge is that you really don't have a lot of deterministic information to go off of. A great many hands never reach the stage where you GET to see other players cards. So the data set is inherently incomplete. You can't know that a player likes to bluff, because an effective bluffer never actually shows his cards. After all, the whole idea of bluffing is that you never make it to the end of the hand where you HAVE to show your cards.So its not a simple problem of looking at past history to determine future play, because the past history is quite incomplete. You can know how often a player bluffs and gets called on it, but there may be another player who is actually bluffing much more often that AI can't know about through simple deterministic means.
This is why players have the option of showing their winning hand after they've won a pot, even if it doesn't make it to the final draw. Actually making it to the final card is not that common in practice (I would guess about 1/3 of hands). A good player will intentionally create the PERCEPTION of a certain playstyle by showing (or not) cards at the end of the hand. They can use that in the future to change their playstyle and catch unsuspecting players. A purely historical approach to data (based on a LOT of unknown outcomes) just doesn't work well here.
Re: No more info than a real player. (Score:2, Informative)
The article did not explain this part of the competition very well. The bots did not have access to any hidden information. The opponent's cards were only revealed at a showdown.
Here's how it works. A mini-match (1000 games) was played, with all of the cards recorded. Then a completely separate mini-match was played with the same cards, but with the positions reversed. This is possible because fresh copies of the programs can be started with a clean slate, having no memory of the previous match. (In fact, the two halves of the mini-match aren't even played on the same computer). Many of these duplicate matches were then played (40 for the fast competition, 12 for the slow competition) between each pair of players.
It is essential to run the matches in duplicate, in order to equalize the opportunities each side gets. The natural fluctation (variance) due to luck is huge in poker, making it very difficult to separate the signal from the noise. The problem is much worse than most people imagine, making accurate measurement of performance a sticky problem. The duplicate matches go a long way toward reducing that variance, and revealing the true long-term expectation.
[On a related note, I created a technique for direct assessment of poker decisions, which cuts away much of the noise. It is described in detail in my PhD thesis, which should be available from the CPRG website (http://games.cs.ualberta.ca/poker/ [ualberta.ca]) in September. A detailed analysis of the AAAI matches will also be available.]
Incidentally, there are a lot of incorrect statements being made in this thread. See the post titled "Some Actual Factual Information" for links to some of the original sources.
- Darse.