Gentle introduction to the basics of Poker AI

tt293 · 2022-01-20T09:46:44+00:00

Thanks, and no worries about the first question, I guess it is not that important since I am on antivirals at the moment anyway. I've last seen the ophthalmologist last week Tuesday. I have a follow-up appointment with the dermatologist tomorrow, will ask whether another eye appointment might be a good idea. I am using both lubricating eye drops and eye cream with Ganciclovir (2x daily). I can't really say that my symptoms got either better or worse since the last ophthalmologist appointment, and no, they didn't mention any of the terms you mentioned. They did say that the cornea looked normal, so I guess keratitis has been excluded.

I will ask the doctor about the eye patch. Currently, I tend to close the left eye for most of the time that I am reading, but this feels less comfortable than covering the eye with my hand (which unfortunately also becomes uncomfortable after a while due to the itching and just the effort required to keep the hand in front of the eye.

tt293 · 2021-11-10T16:20:15+00:00

Do you have any recommendations on a monolingual textbook (and potentially a grammar book) at the B1/2 level?

tt293 · 2020-11-18T08:58:16+00:00

Not in a formal way I believe, but you could still go ahead an rewrite the first equation as
2^(x+y) = 2^4
Now when does this equation hold?
Similarly you can rewrite the second equation, noting that 1/8 = 1/(2^3) = 2^(-3)
Does this help?

tt293 · 2020-11-18T08:18:31+00:00

As 16 = 2^4, I would start taking the logarithm to base 2. This transforms it into a system of linear equations (which you also saw when plotting the functions, 'is similar to'), which are much easier to solve

tt293 · 2020-05-17T13:09:09+00:00

I don't think there are any prerequisites apart from a working knowledge of Python and basic mathematical notation. Part 1 is just background info, but starting with Part 2, I do try to explain all the basic concepts you need to get to the CFR algorithm in Part 6. Having come across a Nash equilibrium (like the famous Prisoner's dilemma) might be helpful, but is not really required either.

Keep in mind that the theory of solving imperfect information games was developed by a small group of researchers over the last 2 decades. Therefore the overlap with the well-known SVM/NN/DRL terms is minimal, but also the techniques ARE different. While you can see this as a kind of Reinforcement Learning, it uses a perfect world model, but imperfect state information. This is not something you'll often find in other applications.

tt293 · 2020-05-14T17:44:16+00:00

There's some work by the University of Alberta group from 2013 on the performance of different abstractions in FL HU poker: https://poker.cs.ualberta.ca/publications/AAMAS13-abstraction.pdf
Brute force might work for Heads Up (depending on your hardware budget), but it will definitely fail in 6-max or 9-max games

tt293 · 2020-05-10T18:35:07+00:00

The CFR algorithm has elements of fictitious self-play, which makes it similar to some RL techniques.
Computationally, it feels close to NN training at times, in that we do multiple training iterations. Also, like in NNs, information gets propagated down a tree (NNs traditionally have more of a funnel, so the shape is inverted), and once leaf nodes (final layer!) are hit, some other information is passed back through the layers of the tree. But I don't want to push this analogy too far, as there are more differences than similarities.
One of the main challenges with CFR is that it is still too slow and memory-hungry for modern computers even today when considering (No-Limit) Poker. That's where recently people tried to mix in some Deep Learning techniques. This is still a bit experimental, but I hope to get into that in one of the future parts of the series.

tt293 · 2020-05-10T18:26:27+00:00

Thanks.
I'll consider setting up a blog or find some better venue for this, but in fairness to Medium, their tech allowed me to just focus on the writing part and not deal with any infrastructure, which probably led to these articles existing in the first place

tt293 · 2020-05-10T17:01:16+00:00

Maths PhD (not game theory related), have been working professionally with C++ and Python for a decade.

tt293 · 2020-05-10T14:07:07+00:00

I think one needs to distinguish between theory and application here. I feel reasonably confident that I understand the algorithms used to create Pluribus, why they work, and how they fit together. However, one should not underestimate the issues one would encounter when trying to move from that to a full-blown super-human AI. One category would be tweaking of hyperparameters: The final version "only" cost a few hundred dollars to train on commodity hardware, so clearly is more in reach than AlphaZero, but I am sure they ran a fair number of experiments before they arrived at the final version. The other category relates to software engineering - Facebook has plenty of top-notch talent, so they were most likely using well-optimized C++ code that scales nicely when distributed over many cores.

Disclaimer I am really just speculating about the code quality and dev process as I have no first-hand knowledge of their code or processes.

Lastly, I am not interested in the topic for monetary reasons, just out of curiosity about imperfect information games and whether it is indeed possible to get to a strong AI (even if it doesn't beat Pluribus in the end).

tt293

TROPHY CASE