Report #003 — do skill verdicts hold across a model release?

Low-resolution: judge quantization (1). These verdicts are supported — in whole or in part — by a case whose old or new with_skill band is zero-width: all 5 judge samples returned the identical score. The judge grades on a coarse quantization grid, so a zero-width band is a clean grid-step effect the 0.05 floor still gates, but one whose finer structure the judge cannot resolve. The verdict stands; its confidence is grid-limited: writing-plans → IMPROVED (1) (interfaces-exact-signatures).

Completed 2026-08-12 on the claude-cli surface — 20 receipts (10 skills × 2 versions, with/without × n=5, judged by claude-haiku-4-5). Metered spend $0 (vendor subscription CLI); estimated metered-equivalent ~$11.79, under the $40 guard. All 20 receipts complete; zero timeouts, zero cases excluded.

v1.1 · 2026-09-14. This entry corrects how the page’s band wording is read; no earlier text is changed, and no figure, verdict token, table value or receipt reference changes. The headline says “2 regressed · 4 improved · 0 mixed · 4 within noise”, the table labels four skills WITHIN NOISE, and the introduction says “Bands that overlap, or separate by less than the floor, are within noise” and “never drift”. None of those four skills had a with_skill case separated under the rule between the two models: no separation detected at the sample size used, which is not evidence of equivalence, not evidence that nothing changed, and not a finding of no drift. The labels are the verdict values this report recorded, and are left as published. As amended:

2 regressed · 4 improved · 0 mixed · 4 with no separation detected over 10 skills.

How to read this page. A case is separated under the rule when its two bands do not overlap and its mean moved by at least the 0.05 effect floor; bands that do not overlap by a smaller move are not separated. What the introduction calls bands that separate by less than the floor are bands that do not overlap. Wherever the page reads a result that was not separated as settled, as within noise or never drift, read it as no separation detected at the sample size used. Wherever it reads a separation as settled, as the low-resolution note’s The verdict stands, read it as a separation detected under the rule, which is not proof that the model’s behaviour moved; that note’s confidence is not a property of any band on this page. Labels and headings are read the same way and are left as published.

The bands. Each band on this page is a mean plus or minus one sample standard deviation, of two kinds: a case’s band, over its five judge samples, and a skill’s with_skill band, over its seven per-case means, which is suite dispersion. The column note’s band (mean ± stddev over 5 judge samples), said of the skill bands, reads as suite dispersion. Each band is a descriptive spread, not a confidence interval, with no coverage probability. The site’s report index and feed quote the amended headline; this page’s summary card, description and social card image keep the headline as published. Filed under the wording rules of the repository's spec 031, amendment A-031-20.

v1.2 · 2026-10-03. This entry adds a dated note under the headline; no earlier text is changed, and no figure, verdict token, table value or receipt reference changes. The note says the headline counts on this page include receipts whose receipt pages mark them Not measured, and that corrected counts follow in spec 159. It was written after each of the 20 receipts this page links was read with the badge's own rule, and all 20 read Not measured.