As of October 2026｜661 million points later: a few words about benchmark cheating

One group in the site's compute records stands out: its reported label is Microsoft Basic Render Driver, its median is 5104, and its maximum is 661036218. That deserves scrutiny. Deliberately submitting invented scores harms everyone using a leaderboard, but attributing an individual result to cheating still requires evidence.

These examples are compute version 1. Groups use the complete renderer string; names are shortened only for display.

A maximum hundreds or over a hundred thousand times the median is a review lead. A reported label does not verify identical hardware.

Separate test versions, retain positive scores, and compare only identical-label groups with at least 20 records. Strictly above 10× or 100× the group median means review, not a deployed cheating verdict.

Of 9264 compared records, 56 exceed 10×, about 0.60%; 20 exceed 100×. Another 193 old-version 3D records are not mixed in. Small groups left out are not thereby certified clean.

Identification, timing, units and implementation can also create outliers. Unknown GPU and WebGL Not Supported do not even locate a verified device. The twelve 3D flags involve eleven labels; the 44 compute flags involve 44. Labels may change, be shared or overlap across tables, so these are not counts of cheaters. Nor do they prove most participants cheat.

Invented results can occupy rankings, distort device comparisons and undermine later data articles. The stored score, label and version alone cannot reconstruct a complete run. A useful reference needs more than an impressive screenshot.

Separate pending-review and verified results while retaining originals; show versions, medians, percentiles and sample sizes; add evidence about timing, work performed and result consistency. These are proposed improvements, not a claim that verification or bans have already been implemented.