How these agents learned Yahtzee

Loowit Play builds games that learn. Both opponents here are neural networks trained from scratch with reinforcement learning (PPO) — they started knowing nothing and played millions of games, improving by trial and error until strategy rose above the noise.

Stage 1 — Curriculum, not chaos

Rather than dropping the agent into full games, training used a 5-phase curriculum: (1) master each single category, (2) the whole upper section and its 35-point bonus, (3) each lower category, (4) the full lower section, (5) the complete 13-turn game. Skills build up layer by layer — you can't play a good full game without first knowing how to optimise a single category. Early phases also use reward shaping (nudges toward good sub-goals) that fades out as the agent matures, so it never gets hooked on the training wheels.

Stage 2 — Adversarial self-play (the league)

Scoring well is one thing; winning is another. The Win Over Everything agent is then trained adversarially: it plays tables against a league of frozen opponents — snapshots of earlier versions of itself plus the expert — and every so often a fresh copy of the current agent is added to the league. This keeps the challenge rising without chasing a single moving target (which makes naive self-play collapse).

Three design choices make it work:

• Reward for placing, not points. A terminal rank reward (win the table → +1, lose → −1) replaces raw score, and a small score weight (α) controls how much it will sacrifice points to win. Too low and it tanks its score for nothing — so we tuned it (α≈0.5).
• It sees the scoreboard. The network takes opponents' totals and its standing as input, so it can gamble when behind and play safe when ahead — the behaviour you can watch on the Styles tab.
• Anchored to the expert. A KL trust-region penalty keeps the competitor close to the score-maximising expert, so it only deviates where competing actually helps instead of forgetting how to play — and it's warm-started from the expert, beginning as a strong player and learning the competitive edge on top.

Two different goals

"Shared dice" — removing luck

When the agents are compared, they draw from the same dice stream, so any difference in score comes from strategy, not lucky rolls. The Winner was trained this way against a league of opponents, learning when to play safe and when to gamble.

The hint button

The "What would the agents do?" button runs each agent's policy on your exact dice and scorecard and shows its recommended move. Watch for spots where Max-Score and Winner disagree — that's the Winner taking a risk to win, even if it lowers its expected score.