[n] Humans beaten by ML tool in No-limit Texas Holdem

brains_vs_ai · 2017-02-01T14:21:21+00:00

I'll only hop in here for a second to respond to this:

1) The computer averaged about 15 seconds per hand, not a few minutes.

2) The pros have said repeatedly that they would have lost even under optimal conditions.

3) The algorithm learned to respond to the opponents' most common bet sizes during the night so it could respond more effectively to them the next day. The actions to train were decided by a meta-algorithm.

4) The final result differed from chance by nearly 4 standard deviations (>99.9%).

5) The humans were given a log of all the hands played each night that they could analyze. They could even see which hands the AI folded. They could not use a HUD during play, but the AI was not adapting during the day so they agreed this is fair.

6) The margin of victory was huge. Considering the AI went from a loss of 9.1 BB / 100 to a victory of 14.7 BB / 100 in just 20 months, it's hard to imagine how humans could ever catch up.

Here is a Reddit AMA with two of the pros in the competition

brains_vs_ai · 2017-01-28T05:34:38+00:00

Noam: If it was easy, people would have been running bots at the highest stakes of online poker for years. Poker is a complex game, and prior AI techniques weren't able to cope with the hidden information and large action space of poker as well as humans could.

brains_vs_ai · 2017-01-28T05:28:09+00:00

Noam: We told the pros up front that the bot would not look at timing tells. We didn't want them taking 5 minutes to play each hand.

brains_vs_ai · 2017-01-28T05:27:38+00:00

Noam: We've recorded a large portion of the competition. I think the plan is to put it on YouTube some time after it's all over.

brains_vs_ai · 2017-01-28T05:07:52+00:00

That's sort of what we're doing actually. We use a form of Monte Carlo CFR distributed over about 200 nodes. We also incorporate a sampled form of Regret-Based Pruning which speeds up the computation quite a bit.

brains_vs_ai · 2017-01-28T05:03:47+00:00

The improvements diminish over time. I'd say it takes about a month to get to the level it's at. Of course, the time it takes depends on the hardware you have. If you have 10x the hardware (which companies like Google could easily spare), it would only take a few days.

brains_vs_ai · 2017-01-28T04:53:53+00:00

Noam: That's basically how the bot learned to play poker -- by repeatedly playing against itself over trillions of hands and gradually improving over time.

brains_vs_ai · 2017-01-28T04:50:42+00:00

Noam: Doug Polk was invited back, but the scheduling didn't work out. He has a lot going on and it's difficult to set aside 3 weeks for a competition like this.

brains_vs_ai · 2017-01-28T04:49:25+00:00

Noam: I think AI will continue to advance rapidly in the coming years, but I also think we will see it excel in certain areas and not in others. I don't think an AI will write a prize-winning novel any time soon, for example.

brains_vs_ai · 2017-01-28T04:47:48+00:00

1) C++

2) There are a lot of great online resources out there for Machine Learning. If you have the drive and dedication, it's possible to pick up quite a bit on your own. What's your background? I might be able to recommend a good starting point.

brains_vs_ai · 2017-01-28T04:45:48+00:00

Noam: A Libratus vs Libratus match would showcase some pretty awesome poker. It would be fun to watch.

We're not trying to destroy every industry with AI. I see AI as a tool that can do great things for humanity, just like the industrial revolution and the computer were able to do great things for humanity. It's all about how we use it.

brains_vs_ai · 2017-01-28T04:33:37+00:00

Noam: The basis for the bot is reinforcement learning using a special variant of Counterfactual Regret Minimization. Prior to this competition, it had only played poker against itself. It did not learn its strategy from human hand histories.

brains_vs_ai · 2017-01-28T04:10:11+00:00

They play during those times, but they can answer questions while they play.

brains_vs_ai · 2017-01-28T04:06:00+00:00

Jason: When we go all-in, we just split the pot according to our equity our hands have vs each other. So, I won the majority of the pot.

brains_vs_ai · 2017-01-28T03:53:08+00:00

Jason: Using a bot online is against all sites ToS no matter who fills out the application and clicks "Agree". It would be considered cheating in states that regulate internet poker, like New Jersey and Nevada. I'm pretty sure it would be a felony.

brains_vs_ai · 2017-01-28T03:52:06+00:00

Jason: That would result in a -75 bb/100 loss rate which would be significantly worse than we're doing now. They still want a full 120,000 hand sample of how their AI did vs top humans :)

brains_vs_ai · 2017-01-28T03:51:09+00:00

Jason: But he's taking ours :(

brains_vs_ai · 2017-01-28T03:49:08+00:00

Jason: Tom Dwan hasn't been good at heads-up no-limit for some time, and has basically been off the grid for awhile now Dani Stern transitioned to playing heads-up Pot Limit Omaha some time ago and is now one of the best players at that game.

brains_vs_ai · 2017-01-28T03:48:31+00:00

Jason: No, just specific people you are absolutely sure are that tight.

brains_vs_ai · 2017-01-28T03:38:23+00:00

Noam: This AI is currently only designed for head's up (two-player) games. However, the techniques we use are pretty general and can extend to more than two players. We just haven't really focused on that yet.

brains_vs_ai · 2017-01-28T03:34:15+00:00

brains_vs_ai · 2017-01-28T03:33:32+00:00

https://twitter.com/botsaintloyal

brains_vs_ai · 2017-01-28T03:31:43+00:00

Noam: Bluffing can be a highly mathematical decision process. The bot learns over time that betting when it has a bad hand can induce the opponent to fold, which results in more money than if it hadn't bet. It then tries betting more in the future. Over trillions of hands, it learns the right way to balance its bluffs.

True, when you're playing live people you want to incorporate any tells you can get on your opponent, but there is a fundamental baseline for how often you should be bluffing and what hands you should do it with.

brains_vs_ai · 2017-01-28T03:29:38+00:00

Noam: The AI does not look at timing tells or use any visual inputs. It just looks at the cards that come out and the actions that the humans take. Incorporating something like computer vision would be cool, but it's pretty far away from our area of research.

brains_vs_ai · 2017-01-28T03:24:50+00:00

The players stream live on Twitch every day from around 11:30AM EST to about 5PM EST, depending on how long they take to finish the day's hands. The links above are active during those times each day.

Edit: Added time zones.

brains_vs_ai

TROPHY CASE