The skill clearly helped: 0.972 with it and 0.618 without, on 3 test tasks.
block, component, search: higher scores with the skill. Compare task scores
Task score bands: block (separated-up): with skill 0.983333 plus or minus 0.028868; without skill 0.575 plus or minus 0.112916; component (separated-up): with skill 1 plus or minus 0; without skill 0.858333 plus or minus 0.139344; search (separated-up): with skill 0.933333 plus or minus 0.028868; without skill 0.421429 plus or minus 0.095119.
A band is the average score plus or minus the recorded spread. These task bands, not the overall averages, determine the result.
0.972 with the skill and 0.618 without; on at least one of the 3 test tasks, the two ranges of scores (the average plus or minus one spread) did not overlap, and the gap was at least 0.05.
Test details and sourcesHow this result was decided
The reader applies the task rule, with a separated-down task taking precedence. Overall means do not determine the verdict. Bands show the mean plus or minus the recorded spread; point bands can reflect score quantization.
block: separated-up. With skill 0.983333 plus or minus 0.028868; without skill 0.575 plus or minus 0.112916.
component: separated-up. With skill 1 plus or minus 0; without skill 0.858333 plus or minus 0.139344.
search: separated-up. With skill 0.933333 plus or minus 0.028868; without skill 0.421429 plus or minus 0.095119.
Three text-only tasks. Driftproof wrote the suite from the pinned skill description and README. These are not author-supplied tests. External tools, generated media and deployed services were not tested.
The eval account's configuration was read and found clean before and after each run, not during it.
This tests the router's written response; it does not test the delegated skill.
The skill clearly helped: 0.978 with it and 0.429 without, on 3 test tasks.
block, component, search: higher scores with the skill. Compare task scores
Task score bands: block (separated-up): with skill 0.95 plus or minus 0.05; without skill 0.433333 plus or minus 0.028868; component (separated-up): with skill 1 plus or minus 0; without skill 0.591667 plus or minus 0.190832; search (separated-up): with skill 0.983333 plus or minus 0.028868; without skill 0.2625 plus or minus 0.094648.
A band is the average score plus or minus the recorded spread. These task bands, not the overall averages, determine the result.
0.978 with the skill and 0.429 without; on at least one of the 3 test tasks, the two ranges of scores (the average plus or minus one spread) did not overlap, and the gap was at least 0.05.
Test details and sourcesHow this result was decided
The reader applies the task rule, with a separated-down task taking precedence. Overall means do not determine the verdict. Bands show the mean plus or minus the recorded spread; point bands can reflect score quantization.
block: separated-up. With skill 0.95 plus or minus 0.05; without skill 0.433333 plus or minus 0.028868.
component: separated-up. With skill 1 plus or minus 0; without skill 0.591667 plus or minus 0.190832.
search: separated-up. With skill 0.983333 plus or minus 0.028868; without skill 0.2625 plus or minus 0.094648.
Three text-only tasks. Driftproof wrote the suite from the pinned skill description and README. These are not author-supplied tests. External tools, generated media and deployed services were not tested.
The eval account's configuration was read and found clean before and after each run, not during it.
This tests the router's written response; it does not test the delegated skill.
