Every claim we can checkone model, one benchmark, more than one published number

We check our own citations, and most of them do not hold. Every row that carries a link of its own was refetched and read: the figure has to appear inside one block of the page that names only this model. 57 of 230 passed, 9 could not be read at all. The largest single failure is Terminal-Bench 2: 104 rows across 35 models whose cited page no longer carries the value recorded. A leaderboard is a living page, so a figure recorded when it was there can be gone by the time you follow the link. The measurements are not thereby wrong. The citations, as printed, no longer support them, and saying so is the entire point of this project.

A ranking is a view over these. Most of the disagreement on this board is not between runners, because for nineteen of our benchmarks only one party publishes numbers at all. It is between CONFIGURATIONS of the same model: on DeepSWE, one model at one runner on one harness scores 1.5 at low effort and 67.2 at maximum. That dial is almost never published beside the score, which is the reason these pages exist.