The skill clearly helped: 0.951 with it and 0.711 without, on 3 test tasks.
transform, exit: higher scores with the skill. Compare task scores
Task score bands: transform (separated-up): with skill 1 plus or minus 0; without skill 0.733333 plus or minus 0.028868; exit (separated-up): with skill 0.878571 plus or minus 0.125357; without skill 0.416667 plus or minus 0.028868.
A band is the average score plus or minus the recorded spread. These task bands, not the overall averages, determine the result.
0.951 with the skill and 0.711 without; on at least one of the 3 test tasks, the two ranges of scores (the average plus or minus one spread) did not overlap, and the gap was at least 0.05.
Test details and sourcesHow this result was decided
The reader applies the task rule, with a separated-down task taking precedence. Overall means do not determine the verdict. Bands show the mean plus or minus the recorded spread; point bands can reflect score quantization.
root: underpowered. With skill 0.975 plus or minus 0.05; without skill 0.983333 plus or minus 0.028868.
transform: separated-up. With skill 1 plus or minus 0; without skill 0.733333 plus or minus 0.028868.
exit: separated-up. With skill 0.878571 plus or minus 0.125357; without skill 0.416667 plus or minus 0.028868.
Three text-only tasks. Driftproof wrote the suite from the pinned skill description and README. These are not author-supplied tests. External tools, generated media and deployed services were not tested.
The eval account's configuration was read and found clean before and after each run, not during it.
The skill clearly helped: 0.917 with it and 0.697 without, on 3 test tasks.
transform, exit: higher scores with the skill. Compare task scores
Task score bands: transform (separated-up): with skill 1 plus or minus 0; without skill 0.733333 plus or minus 0.028868; exit (separated-up): with skill 0.75 plus or minus 0; without skill 0.45 plus or minus 0.070711.
A band is the average score plus or minus the recorded spread. These task bands, not the overall averages, determine the result.
0.917 with the skill and 0.697 without; on at least one of the 3 test tasks, the two ranges of scores (the average plus or minus one spread) did not overlap, and the gap was at least 0.05.
Test details and sourcesHow this result was decided
The reader applies the task rule, with a separated-down task taking precedence. Overall means do not determine the verdict. Bands show the mean plus or minus the recorded spread; point bands can reflect score quantization.
root: underpowered. With skill 1 plus or minus 0; without skill 0.908333 plus or minus 0.124164.
transform: separated-up. With skill 1 plus or minus 0; without skill 0.733333 plus or minus 0.028868.
exit: separated-up. With skill 0.75 plus or minus 0; without skill 0.45 plus or minus 0.070711.
Three text-only tasks. Driftproof wrote the suite from the pinned skill description and README. These are not author-supplied tests. External tools, generated media and deployed services were not tested.
The eval account's configuration was read and found clean before and after each run, not during it.
