Poker Driving Artificial Intelligence Research 212
J-Hawker writes "The Canadian Press has a story about a University of Alberta team that is using Texas Hold-'em to study artificial intelligence. Poker seems to be a much more useful game for this research than chess. From the article: 'Poker has what are currently some of the biggest challenges to (artificial intelligence) systems, and uncertainty is the primary hurdle that we're facing,' said Michael Bowling, adding that the University of Alberta program was able to use its opponents' actions to infer certain things about their hands. 'The same techniques, the same principles that we're developing to build poker systems are the same principles that can be applied to many other problems. The nice thing about chess as a property of the game is what we call perfect information. You look at the board, you know where all the pieces are, you know whose turn it is — you have complete knowledge of the game,' he said. 'But in the real world, knowing everything is just so rare. Everything we do all day long is all about partial information. So poker's much more representative of what the real world's like, and in that sense it becomes a much harder problem.'"
Straight Forward Evaluation (Score:5, Interesting)
What confuses me is how the poker openings differ. I would speculate that a program would be some heuristic relating the ratio of bluffing to "playing the odds." I have gambling friends that play poker all the time and they have these rules that they follow when they play initially against people. They say it's the best until you "know" the people you're playing. Once you can read them then you deviate from the rules. The real irony is that the most successful people I know adhere to a system until they learn someone's movements. Sounds to me like I would write an application that specializes in playing the odds until it recognizes a historical action that statistically reveals the player is bluffing/not bluffing.
Simply put, unless you knew someone's reputation as being a bluffer, you would play the opening hand always the same way. Aren't we forced to program the "AI" of the poker software as being this simple heuristic? Will programs ever be able to "read" players intelligently or will they rely on Markov models & statistics they develop from playing against the same human over and over?
Most unfortunate is the fact that the primary reason my friends gamble is they don't experience the same kind of rush while playing other games as they do with poker because it's more social than other games. If we program applications to beat humans, where does the "social aspect" of the game go?
Even more interesting is the network of poker bots [msn.com] that are set up and running some of the web sites that host poker players. Imagine sitting down at a table of five with four of the other seats taken. Now imagine that these aren't humans but instead bots on four different IP addresses that are sharing card information over an IP connection so that they can leverage odds over you and stop themselves from making stupid mistakes (i.e. they share a card on the table for a pair but really need three of a kind to pose a threat). There's a reason why the percentages fluctuate on TV when cards are revealed whether they be in the flop or in another player's hand.
Re:Straight Forward Evaluation (Score:5, Interesting)
Re:Straight Forward Evaluation (Score:3, Interesting)
The simpler the heuristic used to program the AI, the easier it will be for the opponents to figure out what the bot is doing. A big difference between a mediocre and a successful poker player is the ability to vary their play significantly enough to make it hard for anybody to put them on a hand, without impacting their play so much that they are playing badly.
There are many systems out there developed for the "opening hand", as you call it, and, yes, AI can be programmed to play the preflop game fairly well. After the flop, though, it's a whole different game. As much as you hear about odds in poker, it's not a matter of simple math to calculate them and play "proper odds". You only know your odds if you know exactly what every opponent has... and that's where simple heuristics fail miserably.
Finally, even if you knew everybody's cards, even then you would still need to know exactly how much they are going to bet (if at all) in the future rounds of betting in order to calculate the exact odds you're getting. Once again, that's something that's still very hard for bots to figure out.
Not anytime soon. (Score:4, Interesting)
And even if such software existed, it would basically mean that you couldn't win at online poker anymore because skill would not be relevant anymore. That wouldn't be very different from the current situation with player-versus-casino luck games (like roulette or slots).
And we can all see how poorly these are doing, right?
Already bots playing (Score:4, Interesting)
Limit versus No-Limit Texas Hold'em (Score:2, Interesting)
First, they are playing limit hold'em, which I assume to mean pot-limit texas hold'em. While thats fine, and you'll find plenty pf people that play Pot-Limit, its still a very different game than No-Limit hold'em.
A second thing that I am inferring from the game, is that they are playing heads-up, meaning 1 on 1. Again, this is cool, and I think its a great first step, I still relate that back to Chess. Now if they can take that same AI and play against 8 or 9 other players, effectively, then I'll be impressed.
Re:Straight Forward Evaluation (Score:5, Interesting)
The bot would, ideally, be as good as a very observant player, noting those who bluff and those who don't. Obviously noting 1 or 2 bluffs or non-bluffs would not be enough to make a decision, but over the course of a long tournament, or even better a poker playing career, this information would become very useful. The bot would learn its opponents, and this is what makes it an interesting problem.
I'd argue that cheating at online poker isn't very interesting at all. Humans can do the exact same thing, and online poker companies monitor game's to ensure that there isn't an uncommonly high percentage of people in the same area playing any game. Obviously it might be easier to distribute the bots across the country, but I think it's still more likely (today) to run into actual players grifting you in this manner.
Quantum physics, right? You can accurately determine the odds of winning, or the cards in hand, but not both at the same time? Swear I read something about this somewhere.
Re:Straight Forward Evaluation (Score:2, Interesting)
Second, good poker strategy is not just reactive. It includes active attempts to probe for information by examining the reactions of others at the table. In general your strategy remains fairly constant, but you adjust your behavior (how aggressively do you play), based on the information that you've learned by observing the play of others at the table.
Third, good poker strategy is intentionally deceptive. Sometimes you might specifically decide to play the same hand a different way the second time than the first. Sometimes you will bet your cards for value, and others you won't.
In the long run, you're going to end up with an estimation of each player, and how well/poorly agressively/meekly they're playing, and that's information that you form over time. It's also hard to come by sometimes, because when a player folds, you generally don't get to see their cards, and it can be very expensive to take lots of hands all the way to a showdown just to see what the other player had. At the same time, you're trying to hide as much information as you can from the other players at the table, and possibly create a false image of your intents.
That's what this article is about there is a *lot* there, which makes it a very interesting problem. Your behaviors aren't nearly as fixed as they are in chess. It's further complicated by conflicting opinions as to the best play of certain hands and situations.
If you're interested in the mathematical and game theory aspects of poker, check out "The Theory of Poker" by David Slansky. It has lots of great discussion on the mathematical basis of decisionmaking in poker, including theory of bluffing, etc. Of course, as I just mentioned, it's not the only opinion on how poker should be played, but it's a good starting point.
Poker bots can be mostly sucessful because there are a lot of iditots who play mathematically unsound poker, and are pretty much begging to give their money away. If your bot playes sound poker, it doesn't matter if you give up some money to players who are playing better. Your expectation is positive because there is so much money there to be had.
If you don't like the idea of bots, get some friends, some beer, and play in your kitchen. It makes it much harder to have colluding computers (as you described) take your cash. Plus, then, you also get to drink beer.
Re:Already bots playing (Score:3, Interesting)
Re:Straight Forward Evaluation (Score:5, Interesting)
(1) Knowing the cards of the other players is a small, but significant, advantage. Say you've got two hearts, and your three buddies have a heart each. Well, you're chance of getting another three hearts on the table are significantly affected. (Likewise, if they have none, it increases the chance you'll want to stay in and catch the flop.)
(2) Much more serious, though, is collusion in betting. You and your buddy can conspire to raise the pot *as much as you like*. In a fixed raise game, this is an enormous advantage. Another player cannot just "call" and see the next card, as there will always be a player still to call who can reraise.
Personally, though, I love bots. I'm happy to play them all day long. (So long as they're not colluding, of course...)
Cheers,
Robert
Re:Innovation through vices (Score:3, Interesting)
Considering that a lot of naval technology of the 16th, 17th, and 18th centuries was stimulated by things like warfare, tobacco, sugar/rum, tea, coffee, and the slave trade, is this really surprising?
Computer Go (Score:5, Interesting)
Poker isn't the best! (Score:3, Interesting)
Re:Straight Forward Evaluation (Score:3, Interesting)
That said, uses (and abuses and detection) of out-of-band communication isn't what this research is about; those are concerns for someone else's research project. It's a problem that has plagued poker (and euchre and bridge and a thousand other partial-information games) since the game was invented. That's not a technological problem or a decision making problem, it's a social problem, and A.I. hasn't quite yet advanced to the point where we can worry about it trying to cheat at cards. (Although I welcome the day that we do have to worry about that.)
Limit vs. No-Limit (Score:3, Interesting)
In no-limit poker, when each bet has the potential to cost your opponent all of their money/chips, the decision making process is more critical and mistakes more costly. Variance in no-limit poker is much larger and the AI required to determine whether your opponent is bluffing or has "the nuts" becomes a much larger problem to solve.
Re:Straight Forward Evaluation (Score:3, Interesting)
In general, 4 guys playing together on one table is hard to do more than a couple of times before being flagged on any poker site. So, in most cases, you'll have one buddy telling you that he has no heart (affecting the odds by a negligible amount, and the most likely case), one heart (affecting the odds somewhat, and somewhat less likely to happen), or two (which will happen the least often, but will affect the odds the most).
So, it depends on the definition of 'significant', but the small edge that you will get once every 40-50 hands (probability of having 2 suited cards and flopping a flush draw) is not enough to significantly influence the outcome of a game. The quality of your play in other 39 out of 40 hands will be the deciding factor.
Very true, but this is, luckily, very easily detectable when all of the cards are visible to somebody (i.e. the online poker room operator). So, it's something that you can only do for a short period of time before somebody complains to the operator, at which point it will be obvious that two players playing at the same table a lot are following this particular betting pattern.
It's probably only worth it to sit down at the highest limits possible, try to scoop some money very quickly, and never play again at that site. It's not a very good long-term strategy, though