resources.md

Resources

A non-exhaustive list of repositories, articles and papers I have consulted to put together this project together. To be continually updated.

Git Repositories

• https://github.com/ai-decision/decisionholdem
  • Recently open-sourced solution, though it seems that it doesn't include the code for abstractions, nor depth-limited solving
• https://github.com/matthewkennedy5/Poker -> Really good writing
• https://github.com/fedden/poker_ai
• https://github.com/jneckar/skybet
• https://github.com/zanussbaum/pluribus (An attempt at implementing Pluribus)
• https://github.com/doas3140/PyStack (Python Implementation of DeepStack)
• These students tried to make a copy of Libratus: https://github.com/michalp21/coms4995-finalproj https://github.com/tansey/pycfr (8 years old) -> implementation in CFR, not support nolimit texas holdem
• Pokerbot https://github.com/dickreuter/Poker
• Gym Environment https://github.com/dickreuter/neuron_poker

Blogs

• https://int8.io/counterfactual-regret-minimization-for-poker-ai/
• https://aipokertutorial.com/

Other:

• Really good tutorial by a guy who played 10+ years online poker
• Poker Mathematics Book

Paper links

• An Introduction to CFR (Neller, 2013) ESSENTIAL
• Vanilla CFR
  • (CFR first introduced) Regret Minimization in Games with Incomplete Information (Bowling, 2007)
  • Using CFR to Create Competitive Multiplayer Poker Agents (Risk, 2010)
• Efficient MCCFR in Games with Many Player Actions (Burch, 2012)
• CFR-BR (CFR-Best Response)
  • Finding Optimal Abstract Strategies in Extensive-Form Games (Burch, 2012) (IMPORTANT paper in finding)
• Monte-Carlo CFR (IMPORTANT)
• CFR-D (Decomposition)
  • Solving Imperfect Information Games Using Decomposition (Burch, 2013)
• CFR+
  • (Pseudocode) Solving Large Imperfect Information Games Using CFR+ (Tammelin, 2014)
  • Solving Heads-up Limit Texas Hold’em (Tammelin, 2015)
• RBP Regret-Based Pruning
  • RBP is particularly useful in large games where many actions are suboptimal, but where it is not known beforehand which actions those are
  • Regret-Based Pruning in Extensive-Form Games (Brown, 2015)
• Warm Start CFR
  • Strategy-Based Warm Starting for Regret Minimization in Games (Brown, 2015)
• DCFR (Discounted CFR)
  • Solving Imperfect-Information Games via Discounted Regret Minimization (Brown, 2018)
• ICFR (instant CFR)
  • Efficient CFR for Imperfect Information Games with Instant Updates (Li, 2019)
• Deep CFR
  • Deep Counterfactual Regret Minimization (Brown, 2018)
  • Combining Deep Reinforcement Learning and Search for Imperfect-Information Games (Brown, 2020)

Other ideas

• Depth-Limited Solving (IMPORTANT): This is a key technique that allows us to train a top tier Poker AI on our local computer, by improving a blueprint strategy.
  • Depth-Limited Solving for Imperfect-Information Games (Brown, 2018)
• Abstractions (IMPORTANT): See [[Game Abstraction]]. Abstractions are absolutely necessary, since Texas Hold'Em is too big to solve directly
  • A heads-up no-limit Texas Hold’em poker player: Discretized betting models and automatically generated equilibrium-finding programs
  • Action Translation in Extensive-Form Games with Large Action Spaces: Axioms, Paradoxes, and the Pseudo-Harmonic Mapping (Sandholm, 2013)
  • Evaluating State-Space Abstractions in Extensive-Form Games (Burch, 2013)
  • Potential-Aware Imperfect-Recall Abstraction with Earth Mover’s Distance in Imperfect-Information Games (Sandholm, 2014)
  • Abstraction for Solving Large Incomplete-Information Games (Sandholm, 2015)
  • Hierarchical Abstraction, Distributed Equilibrium Computation, and Post-Processing, with Application to a Champion No-Limit Texas Hold’em Agent (Brown, 2015)
• Subgame Solving: This seems to be impossible to do on a local computer
  • Safe and Nested Subgame Solving for Imperfect-Information Games (Brown, 2017)
• Measuring the Size of Poker
  • Measuring the Size of Large No-Limit Poker Games (Johnson, 2013)
• Evaluating the Performance of a Poker Agent
  • A TOOL FOR THE DIRECT ASSESSMENT OF POKER DECISIONS (Billings, 2006)
  • Strategy Evaluation in Extensive Games with Importance Sampling (Bowling, 2008)

Poker Equity: https://www.pokernews.com/strategy/talking-poker-equity-21291.htm#:~:text=When%20you%20play%20poker%2C%20'Equity,at%20that%20moment%20is%20%2490.

Other Links (Web Pages + Videos)

• https://poker.cs.ualberta.ca/resources.html, this is really good https://poker.cs.ualberta.ca/general_information.html for general information
• Poker Database: https://poker.cs.ualberta.ca/irc_poker_database.html
• The State of Techniques for Solving Large Imperfect-Information Games, Including Poker by Sandholm, really solid overview about abstractions of the game
• Superhuman AI for heads-up no-limit poker: Libratus beats top professionals by Noam Brown
  • AI for Imperfect-Information Games: Beating Top Humans in No-Limit Poker by Noam Brown at Microsoft Research

Poker Agents Papers

• Slumbot "250,000 core hours and 2 TB of RAM to compute its strategy"
• Polaris (2008)
• Baby Tartanian 8 (2016) "2 million core hours and 18 TB of RAM to compute its strategy"
• DeepStack (2017)
• Libratus (2017)
  1. Blueprint Strategy (Full-Game Strategy) using MCCFR
  2. Subgame Solving with CFR+
  3. Adapt to opponent
• Pluribus, video here (2019)