17 AIs, 7,700 hands of poker: frontier vs cheap vs local vs Jev [P]

Explain Like I'm Normal

A massive independent study tested 17 different language models across 7,700 hands of poker to benchmark reasoning and bluffing capabilities. OpenAI's o1-preview dominated the field with superior strategic depth, while smaller and local models struggled significantly with game theory and long-term planning. The results provide a unique look at how different architectures handle hidden information and adversarial decision-making.