Strength
Two independent measurements against GNU Backgammon
The claim is narrow and it is the only one we make: on cubeless checker play, Raccoon’s network is stronger than GNU Backgammon’s network at the same search depth — none. Two measurements support it, one on a fixed benchmark of scored decisions, one from 18,000 games of actual play.
1. The BGSage money benchmark
A fixed corpus of 500 self-play money games. Every non-trivial decision in it carries a reference cubeless equity computed at adaptive precision — the closest decisions get full rollouts, the rest get deep static evaluations — so an engine can be scored on how much equity it gives away per decision rather than on who won.
The benchmark data was published by BGSage (repository, bot performance study), and this project is in their debt for making it available. Scoring one engine against another honestly is otherwise the most expensive part of the work.
| Engine | Search | PR (n = 14,693) | Blunders |
|---|---|---|---|
| GNU Backgammon | full-width, 2 ply | 0.56 | 11 |
Raccoon exp018-ep22 |
none — one network evaluation | 0.95 | 19 |
| GNU Backgammon | none — one network evaluation | 2.14 | 95 |
PR is mean move-selection error × 500 — mean(max(0, best_equity − chosen_equity)) × 500 — so lower is better and zero is perfect. A blunder is a single decision costing more than 0.08 points. All three engines were scored on the same 14,693 checker decisions, with cubeless references at three precision tiers: 5,695 rollout, 2,842 three-ply-team, 6,156 three-ply. The corpus’s 2,842 cube decisions are excluded, because none of these engines is being scored on the cube. These figures are not comparable with BGSage’s own published PR numbers, which are cubeful and do include cube decisions.
One caution on our own number: 0.95 is the score of this particular checkpoint, picked because it was the best. The same training recipe averages PR 0.99 (SD 0.023) across epochs 15–24, and that is the fairer reading of the method’s level. Either number is well clear of 2.14.
2. Head-to-head, 18,000 games
A benchmark scores decisions in isolation. This measures what happens when the two engines simply play each other: cubeless money game, seats alternated, both sides picking moves with a single network evaluation.
| Measure | Result | 95% CI | Games |
|---|---|---|---|
| Points per game vs GNU Backgammon 0-ply, variance-reduced | +0.047 | ± 0.003 | 18,000 |
| The same games, plain unadjusted average | +0.048 |
± 0.020 | 18,000 |
The two rows are the same games read two ways. Backgammon results are dominated by the dice, so the plain average is a blunt instrument: after 18,000 games it still pins the edge down only to within ± 0.020 points per game — enough to put Raccoon ahead, not enough to say by how much. The variance-reduced row subtracts the luck in each roll — a control-variate technique standard in GNU Backgammon’s and XG’s rollouts — which is worth about 54× in variance here. Matching that confidence interval by counting points would have taken roughly 970,000 games. How the variance reduction works, and why it is unbiased
The engine you play here is the engine that was measured
The browser runs the same network exported to ONNX, and that exported file was re-scored on the same benchmark at the same sample size: PR 0.9500 for the ONNX file against 0.9498 for the PyTorch checkpoint. Nothing was cut down to fit in a browser tab.
What this does not claim
Checker play only, and cubeless. No doubling cube, no match play, no cube decisions of any kind — a strong cube handler could beat this engine comfortably in a real match while being worse at moving checkers. Both are milestones on the roadmap, and neither has been measured because neither exists yet.
Against GNU Backgammon, not against everything. GNU Backgammon is the strongest freely available engine and the natural reference; the commercial engines are closed, and we have not benchmarked against them.