Did the AI change?

The same popular skills and the same tests, run again on the same model on a later date. Here is what Driftproof's comparison read for each pair.

What this is

People often say their AI assistant changed under them, and nobody can tell them whether it did. This page is a first reading of one way to find out. Driftproof keeps a receipt for every test it runs: the skill's exact text, the test tasks, the model, every score and a seal. Running the same skill and the same tests again on the same model on a later date gives a second receipt. Driftproof then compares the two with the same rule it uses for every comparison and says what it read.

A canary is an early warning: a small, fixed check repeated over time, so that a change shows up as a difference between two readings of the same thing. Here the canary is a handful of popular skills, each with its own small set of text tasks, written by Driftproof and frozen before the first run.

This page is not a status page, and it promises no schedule: we decide when to run it again. It is not a verdict about Claude or about any model. A difference between two dates is read under a fixed rule and is never attributed to a cause.

How the skills were chosen: The first five skills in Report 018's fixed order that do real work rather than hand off to another skill, chosen before any result: grill-me, grill-with-docs, improve-codebase-architecture and tdd from mattpocock/skills at 49dd158d, and hyperframes-cli from heygen-com/hyperframes at c4440dfb. Each was run with its exact skill text and its frozen tests, and both were checked against the earlier receipt's.

How to read this page

A separation between two dates is detected under the rule, never attributed. Between the dates any of these could have moved the scores: the model behind the name, the harness that ran it, Driftproof's own version and scoring code, the judge model behind its name, the judge's own variation and chance. This page does not say which one made the difference.

Where the judge is the same model as the one measured, a change in that model also changes the scoring, so the two cannot be told apart.

No separation is not evidence of sameness: it means no test task separated under the rule at this sample size.

Chance alone can produce a separation. Each task's band comes from a few answers, so the more tasks are read, the likelier one separates with nothing changed.

No verdict is a result: where Driftproof refused to compare two receipts, or a receipt did not measure every task, or a separation did not hold for every score the lost answers could have had, it says so and asserts nothing.

A few tasks per skill on a few dates is a first look, not a trend.

In the rows, the harness is the program that ran the model. In the printed readings, a case is one test task, a band is the average plus or minus the usual spread of the scores, and the effect floor is the smallest gap that counts.

Driftproof's comparison prints a label in capitals before each reading. This page writes that label in plain words and keeps the rest of the reading as printed. The labels and their plain words:

NO SEPARATION DETECTED

No separation detected

DRIFT

A separation detected on a task, with the skill lower on the later date

IMPROVED

A separation detected on a task, with the skill higher on the later date

MIXED

A separation detected, lower on some tasks and higher on others

INCONCLUSIVE

Inconclusive

NOT MEASURED

Not measured

REFUSED

Refused

Do this for your own file

In Claude Code, type /driftproof:recheck and the path to your file. The first use saves a copy of your file, its tests and its run settings and runs nothing. Each later use runs that saved copy and keeps another dated receipt, and a recheck compares each new result with the previous one and with the first saved reading. It cannot compare with a date before you saved.

From a terminal, the same steps are driftproof recheck init --trusted-skill (only for a file you wrote yourself), driftproof recheck run --trusted-skill on each later date (you give that flag each time; the saved copy does not keep it) and driftproof recheck history for the comparisons.

This checks Claude through Claude Code with fixed tasks, not anyone's chat history.

Every pair of receipts this site holds for the same skill, the same tests, the same model, the same judge model and the same setup (how each answer was recorded and which sides of the comparison ran) on different dates is listed on The same skill on different models and dates.