Driftproof
Menu
Skills.sh snapshot 2026-10-17: text tasks
Worth installing?

find-skills

Here is what happened when we tested its written answers, with and without the skill.
About this test

find-skills: Skills.sh snapshot 2026-10-17: text tasks by Driftproof; Haiku 5.5 judge; eval account checked before/after runs, not during

Report 015: Skills.sh snapshot 2026-10-17: text tasks

Skill skills/find-skills in vercel-labs/skills.

Each result has its own test record. Results are not combined.

Measured results

Both model sets use Haiku 5.5 as judge.

Driftproof measured

Answering model: Opus 5.5. Judge: Haiku 5.5. Date: 2026-10-18.

Three text-only tasks: Clearly hurt.

At least one task got worse. That takes priority over the overall average.

Read test record

Why this result?

The skill clearly hurt: 0.889 with it and 0.891 without, on 3 test tasks.

search: lower scores with the skill. Compare task scores

Task score bands: search (separated-down): with skill 0.75 plus or minus 0.05; without skill 0.92 plus or minus 0.057009.

A band is the average score plus or minus the recorded spread. These task bands, not the overall averages, determine the result.

0.889 with the skill and 0.891 without; on at least one of the 3 test tasks, the two ranges of scores (the average plus or minus one spread) did not overlap, and the gap was at least 0.05.

Test details and sources
How this result was decided

The reader applies the task rule, with a separated-down task taking precedence. Overall means do not determine the verdict. Bands show the mean plus or minus the recorded spread; point bands can reflect score quantization.

  • search: separated-down. With skill 0.75 plus or minus 0.05; without skill 0.92 plus or minus 0.057009.
  • missing: separated-up. With skill 0.95 plus or minus 0.05; without skill 0.77 plus or minus 0.115109.
  • scope: underpowered. With skill 0.966667 plus or minus 0.028868; without skill 0.983333 plus or minus 0.028868.

Three text-only tasks. Driftproof wrote the suite from the pinned skill description and README. These are not author-supplied tests. External tools, generated media and deployed services were not tested.

The eval account's configuration was read and found clean before and after each run, not during it.

This tests the router's written response; it does not test the delegated skill.

Read receipt · Receipt JSON · Suite · Retained answers

Earlier attempts
Receipt metadata

Attempted generations: 22. Measured generations: 22. Lost draws: 0. Judge samples: 110.

Model
claude-opus-5-5
Judge
claude-haiku-5-5
Repository
vercel-labs/skills
Skill path
skills/find-skills/SKILL.md
Pinned commit
13e4063a1cf913f5606d57d42ab83a86f5001e04
Harness
claude-code 2.1.287
Receipt hash
ae40fa3f5a12a6d72d93f6ebfc0a456757297886a88224e942022166aee945b4
Plan hash
af83e8024b571620361e0e1ee1e371c21589e6788f25bbe53542fd11f941da53

Driftproof measured

Answering model: Sonnet 5.5. Judge: Haiku 5.5. Date: 2026-10-18.

Three text-only tasks: Clearly hurt.

At least one task got worse. That takes priority over the overall average.

Read test record

Why this result?

The skill clearly hurt: 0.853 with it and 0.944 without, on 3 test tasks.

search: lower scores with the skill. Compare task scores

Task score bands: search (separated-down): with skill 0.75 plus or minus 0; without skill 0.95 plus or minus 0.05.

A band is the average score plus or minus the recorded spread. These task bands, not the overall averages, determine the result.

0.853 with the skill and 0.944 without; on at least one of the 3 test tasks, the two ranges of scores (the average plus or minus one spread) did not overlap, and the gap was at least 0.05.

Test details and sources
How this result was decided

The reader applies the task rule, with a separated-down task taking precedence. Overall means do not determine the verdict. Bands show the mean plus or minus the recorded spread; point bands can reflect score quantization.

  • search: separated-down. With skill 0.75 plus or minus 0; without skill 0.95 plus or minus 0.05.
  • missing: underpowered. With skill 0.86 plus or minus 0.065192; without skill 0.883333 plus or minus 0.116905.
  • scope: underpowered. With skill 0.95 plus or minus 0.070711; without skill 1 plus or minus 0.

Three text-only tasks. Driftproof wrote the suite from the pinned skill description and README. These are not author-supplied tests. External tools, generated media and deployed services were not tested.

The eval account's configuration was read and found clean before and after each run, not during it.

This tests the router's written response; it does not test the delegated skill.

Read receipt · Receipt JSON · Suite · Retained answers

Earlier attempts
Receipt metadata

Attempted generations: 24. Measured generations: 24. Lost draws: 0. Judge samples: 120.

Model
claude-sonnet-5-5
Judge
claude-haiku-5-5
Repository
vercel-labs/skills
Skill path
skills/find-skills/SKILL.md
Pinned commit
13e4063a1cf913f5606d57d42ab83a86f5001e04
Harness
claude-code 2.1.287
Receipt hash
e2403573b7370d33e8b7d18f8287d7a90abe1ed18feb1cab482baa909f0b7b67
Plan hash
af83e8024b571620361e0e1ee1e371c21589e6788f25bbe53542fd11f941da53

How we test a skill

A skill gives an AI extra instructions. We ask: do those instructions improve its written answers?

  1. The task Same question. Same AI.

    Driftproof writes tasks from the skill description.

    Give the AI a task
  2. The comparison Change the instructions
    With the skill Written answer
    Without the skill Written answer

    Repeat both ways to see how much answers vary.

  3. The result Judge each task
    Haiku 5.5 scores the answers

    A clear drop on any task takes priority. Otherwise, a better task can mean the skill helped. When answers vary too much, we cannot settle the result.

These tests cover written answers, not installation, tool use or how useful a skill is in general.

Receipts by Driftproof (driftproofhq/driftproof), licensed Apache-2.0.

Disagree, or want a re-run?

Built on from committed receipts.