Glossary
Ten words this site uses in a particular way, two sentences each, every one linking to the section it comes from. The verdict rule, in the methodology's own words: Bands that touch or overlap, or moves below the floor, are reported as within noise, never as drift.
- Drift report
- A dated comparison of two runs of the same eval suite, asking whether a skill still helps after something underneath it moved. It names what moved, reports a verdict per cell, and links the receipts every figure came from. Source: the drift verdict rule.
- Receipt
- The dated, hash-verified JSON file a single run emits: the skill and its content hash, the model, the surface, every case score, and the comparison between the two arms. It is the unit of evidence on this site, and every published number traces to one. Source: the receipt specification.
- Band
- The range a score carries instead of a single number, because a scorer asked twice does not answer identically twice. A result is a band, not a point, and the width of that band is what decides whether a difference is real. Source: with vs without, sampled, banded.
- Effect floor
- The smallest lift that counts as a lift, below which a difference is reported as no effect however the bands fall. It exists because a score can move by a rounding artefact of the scoring grid and mean nothing at all. Source: the drift verdict rule.
- Substrate
- The model underneath the skill: a specific version, from a specific provider, at a specific tier. Driftproof exists because the substrate moves while the skill file stands still. Source: providers, surfaces and neutrality.
- Surface
- How the model was reached: a metered API or a subscription CLI. The same model over two surfaces is not guaranteed to behave identically, so every receipt records which one it ran on. Source: providers, surfaces and neutrality.
- Refusal
- A published result that reports no verdict, because the run could not stand behind one. A refusal is an outcome and not an error, and publishing it is the alternative to guessing. Source: the drift verdict rule.
- Cell
- One skill measured on one substrate: the smallest unit a report gives a verdict to. A report is a grid of cells, and a cell that refuses does not sink the ones that did not. Source: with vs without, sampled, banded.
- Arm
- One side of the comparison: the run WITH the skill, or the baseline run without it. Every case is run in both arms, and the difference between the two arms is the only thing a lift can mean. Source: with vs without, sampled, banded.
- Lattice
- The four verification levels a receipt can carry: UNVERIFIED, DECLARED, TESTED and FORMAL. A number Driftproof measured itself is TESTED; a number imported from another tool is DECLARED, recorded but not verified, and never given a pass or fail verdict. Source: receipts are an open format.