Report 008: the skill stabilises the floor, not the ceiling

HOLDS ACROSS THE RELEASE, ON A PAIR WHERE THE MODEL IS THE ONLY THING THAT MOVED.

What that does and does not say. It says the two skills' measured behaviour on claude-fable-5-1 is indistinguishable, by this instrument, from their behaviour on claude-fable-5, and that the attribution is clean: skill content_hash and suite_hash were asserted identical against the Report 007 baselines before the first call, and surface, judge and judge samples per case are held on both sides of both pairs. It does not say the skills got better, it does not say they got worse, and it does not say nothing changed underneath. A within-noise verdict is a statement about what this instrument can resolve at this sample size, which is 92 generation draws across 28 arms.

Where the verdict is closest to its own edge. Exactly one case in the run is held inside the verdict by the effect floor rather than by band overlap: writing-plans@claude-fable-5-1's file-structure-by-responsibility, whose bands ARE separated (0.831 ± 0.006 (generation) against 0.853 ± 0.013 (generation)) and whose move of +0.023 is below the 0.05 floor. It is named here rather than left in a table, because it is the one row where a reader who set a lower floor would read a different verdict. Every other case in the run fails the separation test outright, so the verdict does not rest on the floor.

Report 007 found that across three cells the arm that would not sit still was the baseline, and that what the skill did was stabilise the floor rather than raise the ceiling. That report moved the instrument and held the substrate. This one moves the substrate and holds the instrument, and the same shape comes back.

The headline case of code-review-and-quality@claude-fable-5-1 went pass to borderline, and the reason is not the skill. severity-labeled-findings carries 96.2% of that cell's entire lift: its +0.488 against a sum of +0.507 across all 7 cases, with the remaining six summing to +0.019. Take it out and the cell's lift is +0.003, indistinguishable from zero and far below the 0.05 floor. On the release axis that one case moved like this:

The baseline arm fell 0.585 to 0.308. The with_skill arm held, moving -0.121 and staying inside its own band. The cell's lift on this case therefore grew, +0.332 to +0.488, and almost all of that growth is the unskilled arm getting worse rather than the skilled arm getting better. A report that quoted the lift without this sentence would credit the skill with something the model did to the arm that does not have it.

The stability moved the other way, and that is the cost of the same fact. The with_skill band on that case widened from ± 0.026 to ± 0.147, a factor of 5.7, which is what took its outcome label from pass to borderline and what makes the release-axis verdict on it read within noise despite a move of -0.121. The receipt says where the width came from: the arm drew a fourth generation with stopping_reason: "stabilised", and one of the four scored far below the others. A one-case pillar that has become unstable is a fragility this page discloses whatever the aggregate verdict says.

The same pattern carries writing-plans@claude-fable-5-1, with two pillars instead of one. The 2 cases whose baseline arm outright fails carry 88.6% of that cell's lift: full-small-plan-header-and-tasks at +0.480 (baseline 0.415) and task-right-sizing-testable-deliverable at +0.302 (baseline 0.449). The other 5 cases sum to +0.101. In both cells, on both models, the lift is not spread across the suite: it is a handful of cases where the unskilled arm fails and the skilled arm does not.

Which arm actually moved between the two models. The differ compares the with_skill arm only, so baseline drift is invisible in its output. Computed here over the same receipts, the largest movements in the run are baseline movements:

Across all 14 cases the widest baseline movement is 0.277 and the widest with_skill movement is 0.121. The floor moved further than the ceiling did, on a release the skill text did not know about. That is Report 007's finding, reproduced on a substrate change rather than an instrument change, and it is the reason this report presents the aggregate lift with its per-case table attached rather than on its own.

One honest caution about the sentence above. The baseline arm is where a model's own default behaviour shows through, so it is the arm most exposed to a release, and finding the larger movement there is not by itself surprising. What the two reports together support is narrower than a law: on five cells across two model pairs, the arm with the skill text moved less than the arm without it, and the skill's measured value came from the cases where the unskilled arm failed. Five cells is five cells.

Every dollar below excludes cached input tokens. The receipts price all input at the list input rate, cached included, and record cached_tokens per draw precisely so a reader can re-price. On this surface cached context is 90.1% to 92.2% of each call's input here, so on the list-rate basis the largest term in every mean is harness preamble that neither arm chose and the skill cannot move. Report 007 v1.1 is what happened when a mechanism was read out of that basis: two of its three cells changed the sign of their incremental under this same re-pricing. This page therefore prints one basis and does not claim the other.

The incremental is positive in both reports on writing-plans@claude-fable-5-1 and only on that cell. code-review-and-quality@claude-fable-5-1 is -$0.028848 on the older model and +$0.012699 on the newer one, so it changes sign between the reports on the fresh basis. A single sentence covering both cells would be wrong about one of them, and the difference is small enough in absolute terms that neither sign should be leaned on: the whole column is hundredths of a cent per call.

What moved in the tokens, writing-plans@claude-fable-5-1. The with_skill arm's per-call cost rises between the two models, and 77.3% of that rise is cached context priced at the list rate. 22.7% of it survives on fresh input, and it is chiefly output: +1,635 mean output tokens per call, at an output rate 5 times the input rate, against +2,177 fresh input tokens. That comparison is at an identical draw allocation: 24 draws in both reports, allocated 3,3,3,3,6,3,3 both times, with the same case in the wide slot. Adaptive stopping made the same decisions on the new model in this arm, so the movement is per-draw token growth and not a change in what was averaged. claude-fable-5-1 writes materially longer plans for the same seven cases.

And on code-review-and-quality@claude-fable-5-1, the same arithmetic points the other way. Its with_skill arm emits -264 output tokens per call on the new model and takes on +244 fresh input tokens, so on fresh input that arm gets cheaper per call across the release while the list-rate basis says it gets dearer. Longer outputs are a property of one of these two skills' generations on claude-fable-5-1, not of the release.

The caveat this whole section rests on, stated plainly. Cached input is not free to the vendor and is not billed at the list rate either; the receipts over-estimate it and this page under-estimates it by excluding it entirely, and neither basis is the true one. Actual metered spend for this run was $0.00: the surface is a subscription CLI and every figure here is the if-it-had-been-metered equivalent at each receipt's own frozen pricing snapshot. Prices are identical across all four receipts (10 in / 50 out per Mtok for both Fable models), so nothing above is price-driven. Judge cost is measurement overhead this project imposes and is excluded from every field. No figure on this page comes from the run's pre-flight projection or from any cap.

Latency, observed on subscription CLI surface, indicative: inside this run a call is slower with the skill than without it, by +1262 ms of median wall-clock on code-review-and-quality@claude-fable-5-1 and +2740 ms of median wall-clock on writing-plans@claude-fable-5-1. Wall-clock here includes cold starts and the vendor's own harness and is not a measurement of serving latency.

Launched against claude-fable-5-1 on 2026-09-01, from an isolated git worktree at 1b7b151 on spec/019b-fable-5-1-registration. code-review-and-quality ran 2026-09-01T18:45:54Z to 2026-09-01T20:44:42Z, writing-plans ran 2026-09-01T20:44:42Z to 2026-09-02T01:14:17Z, 258 calls and 294 calls, both exit 0. The receipts landed at a0aecb0.

The hashes were asserted before the first call, not after the last one. A pre-launch assertion exited 0 having compared, per cell, the skill content_hash and the suite_hash the run was about to use against the Report 007 baseline receipt it would be diffed against: code-review-and-quality (13d360d7f786, 4729eb8fc298, 7 cases) and writing-plans (5325ab3d55d5, 37cb3c341572, 7 cases). Both matched on both fields. That ordering is the whole attribution claim. Checking afterwards tells a reader what was measured; checking first is what makes it a controlled comparison, because a mismatch would have stopped the run before it spent anything. The differ re-confirms the same equality from the receipts themselves, so the assertion has a second, independent witness.

Zero draws were lost to timeout. 92 generation draws across 28 arms, 92 measured, 0 unmeasured. The per-call timeout was 300,000 ms, resolved from the surface policy rather than passed as a flag (lib/provider.js retryPolicyForSurface). Report 007's subject was that this policy had been declared and not executed since 2026-07-27, shadowed by a numeric literal at the call site, so every run before the fix ran at 120 s; this is the first release-drift run to execute under the policy as written, and the loss count is the control on that fix rather than a claim about this run's difficulty. A restart script was prepared before launch and never invoked.

Caps per cell: --max-cases 7, --max-calls 840, --max-usd 11. Judge claude-haiku-4-5 at 5 samples per draw on both cells. Surface claude-cli throughout, metered spend $0.00. Verification level TESTED on all four receipts, none carrying a run.source, so none is an imported declaration and the differ's NOT MEASURED path was not reached. The timings, the assertion result and the timeout policy above are transcribed from the run log into reports/report-008-run-record.json, which records that log's sha256 so the transcription is checkable; the log is an operator-home file and is not committed.

• Two cells is two cells. Fourteen cases on one substrate family is not a corpus, and nothing here generalises past the two skills named on it.
• A within-noise verdict is not a finding of no change. It is a statement about what this instrument resolves at 92 draws across 28 arms, and a real move smaller than these bands would look exactly like this.
• One case is held inside the verdict by the effect floor alone, named in the Verdict above. A reader who set a lower floor would read that row differently.
• The aggregate lift in both cells is not band-verified, and the report claims no aggregate improvement from it. What is shown per case is shown per case.
• The lift in both cells is concentrated in a few cases, and in code-review-and-quality@claude-fable-5-1 one case carries 96.2% of it. A concentrated lift is more fragile than a spread one, and that case's band widened on the new model.
• The differ compares the with_skill arm only. The baseline movements this report leans on are computed on this page from the same receipts, not emitted by a shipped command, which is the same disclosed gap Report 007 filed.
• One case is bimodal rather than merely noisy. task-right-sizing-testable-deliverable drew 6 times with a variance ratio above 1, its draw means splitting into a high and a low group. A single mean ± sd understates what a user of that case would experience, and it was the widest case on the older model too, so it is a property of the case rather than of the release.
• The economics rest on a basis choice, disclosed above, and the sign of an incremental on this surface depends on it.
• run.model_release_date is null in both receipts for a report whose axis is a release. The date is asserted in the launch script and the run record, not in the receipt, which is a schema gap this run surfaces and does not fix.
• Every verdict is verifiable from the receipts, which are linked below rather than described.