The invisible space that AI graders got backwards

Below are two answers to the same coding question. One is right. We showed them to the Claude graders inside three popular open-source evaluation tools and Claude Code's own eval command, and they usually failed the right one and passed the wrong one, each time with a plausible-sounding reason. Try it yourself before you read on.

Found by Driftproof (driftproofhq), 27 October 2026

Which answer is correct?

def accepts(s: str) -> bool:
    return s.strip(" ") == "42"

assert accepts("42") is True
assert accepts(" 42 ") is True
assert accepts(" 42 ") is False
assert accepts("43") is False

def accepts(s: str) -> bool:
    return s.strip("  ") == "42"

assert accepts("42") is True
assert accepts(" 42 ") is True
assert accepts(" 42 ") is True
assert accepts("43") is False

If you picked B, you're in good company. Both answers' own tests pass when you run them. The difference is a character you can't see: a no-break space, the one typesetters use to keep "28 October" or "Haiku 5.5" on one line. This page has several of them in its own text, which is perfectly normal. Flip the X-ray switch above to find them. In the quiz, one of them decides the answer.

What the graders did

Many evaluation tools use a second AI as the grader: it reads an answer and decides pass or fail. We gave these two answers to the graders inside three popular open-source tools, LangChain's OpenEvals, Promptfoo and NVIDIA's SkillEvaluator, with Claude Haiku 5.5 and Claude Sonnet 5.5 doing the grading. We ran each tool's real grading code, routed to Claude through Claude Code rather than the tool's own Anthropic connector.

In the explanations we read, a common pattern was this: the grader saw two tests in A that seemed to check the same thing and expect opposite results, and decided A contradicted itself. Promptfoo's grader, with Sonnet, rejected A because its tests used "identical visible inputs", and said running them would raise an error. Running them works fine. OpenEvals' grader, with Haiku, passed B, describing the characters B strips as ordinary spaces and dismissing the third test as a duplicate.

The one grader that got every as-written trial right was different. Anthropic's skill-creator grader, driven by the same two Claude models, is allowed to run code. It checked the actual file by running it, and the computer, unlike a model reading the text, could tell the two spaces apart.

How Driftproof found it

Driftproof measures whether the instructions people give AI coding assistants actually make them better. Before any paid run, we do a small practice round to check our own grader. In the practice round for our next report, one coding question turned on this exact distinction.

Our graders, Claude Haiku and Claude Sonnet, got it backwards every time: 0 of 6 correct calls each. They failed the correct answer and passed the wrong one. When we added a list of the unusual characters after the answer, Sonnet got 6 of 6 and Haiku 5 of 6. In a later run with a fuller list, Haiku got it wrong again, so our grader is now Sonnet, and we score by running tests. Everything else in the same run, including the ordinary-space cases, was graded correctly. The problem was this one character.

We checked whether Claude Code itself drops the character. In a captured request from the command-line tool, it arrived byte for byte. That rules out the obvious culprit, though we haven't yet captured our own original pipeline end to end. We then rebuilt the question from scratch, the version in the quiz above, to see whether the tools other people use behave the same way.

Three tools, the same mistake

Correct calls. Each tool graded both answers three times with each model. NVIDIA's grader with Sonnet failed both answers rather than flipping them; with Haiku, two errors remained even with the list. *Two of skill-creator's twelve replies with the list broke its output format and are not counted. Every tool also graded two simple control answers without the special character correctly, 16 of 16 in all.

We also read how eight popular evaluation tools hand an answer to their grader. In the grading paths we read, none adds a note pointing these characters out. A few trim the start and end of the text first, which can delete such a character when it sits at an edge.

Anthropic's own eval command has the same blind spot. Claude Code's built-in claude plugin eval grader, judging the same two answers, got 0 of 6 right with Haiku 5.5 and 1 of 6 with Sonnet 5.5. We then wrote the identical code with the no-break space spelled out as \u00a0, which Python reads as exactly the same program. Both graders got 6 of 6. The only thing that changed was whether the grader could see the character. Anthropic's skill-creator grader, which runs the code, got it right; this one only reads it. (claude-code #105497)

The fix: say what's there

The fix is deliberately small. Alongside the answer, after it, a short note is added that lists any unusual spaces and invisible characters, with their exact position. For answer A it reads, simplified:

Character facts (not a verdict):
line 6, column 17: U+00A0 NO-BREAK SPACE
line 6, column 20: U+00A0 NO-BREAK SPACE

The answer itself is never changed, and the note doesn't say the character is wrong, because sometimes it is exactly right. It only stops the grader mistaking one character for another. In all three proposed fixes it is opt-in, so nobody's existing results change unless they turn it on.

Did it hold up?

We tested the actual fix, as submitted to OpenEvals, through OpenEvals' own code:

A general warning about Unicode helped little: Haiku stayed at 0 of 6 and Sonnet reached 3 of 6. The exact list got both to 6 of 6. We also ran 72 control checks, including cases where unusual characters are legitimate and must be kept, and all 72 stayed correct. In these controls, the list caused no false alarms.

In NVIDIA SkillEvaluator we checked whether it moves a real score. On a separate but similar question, two answers generated fresh by Claude both wrongly accepted the no-break space. NVIDIA's unmodified verifier passed both answers with both graders, with overall scores of 1.00 (Haiku) and 0.99 (Sonnet). With the list, all four false passes became correct rejections and both scores dropped to 0.75. A general warning left all four passes in place. That was one grading per combination, on replayed answers rather than a full benchmark run.

It is not a cure-all. In a wider test with ten different tricky questions, written in JavaScript with different prompts, the graders got most cases right even without the list. The list fixed Haiku on the no-break space question, but it also made Haiku pass one answer that wrongly deleted a different invisible character, a case it had graded correctly without the list. And Sonnet often failed to reply in the required format, most of all with the list: 27 of 60 replies were unusable. The issues linked below set out these limits.

Before you reply

"Just normalise the text first." Compatibility normalisation (NFKC), or collapsing whitespace, turns the no-break space into an ordinary space, and then the wrong answer looks right. On a question like this one, that throws away the evidence.

"A human can't see it either." Exactly. That is why a model reading text is the wrong tool for this check, and why a simple list from code helps so much.

"So coding benchmarks are broken?" No. The big ones, like HumanEval and SWE-bench, grade by running code against tests, and tests catch this when they include the character. It matters where an AI grader reads the code, as in MT-Bench, where 10 of the 80 questions are coding questions graded by a model.

"How often does this happen?" We don't know how often it changes a grade. The characters themselves are rare: none in HumanEval or MBPP, and no no-break spaces in 2,415 chatbot answers from AlpacaEval. Translation is the exception: 39 of 300 French translations in one real dataset contained them.

"Is this just Claude?" We tested Claude graders only. Other models may do better or worse.

What we changed

For our next report, Driftproof grades coding answers mainly by running them against hidden tests, and those tests include the tricky characters themselves. The AI grader only covers what a test can't check. It is also what Anthropic's own guide to building evaluations recommends for coding tasks.

This is why Driftproof runs a practice round before every paid run: to catch the instrument being wrong before it produces a number anyone relies on.

Where the fixes stand

Status on 27 October 2026. Each fix links to an issue (NVIDIA #241, OpenEvals #292, Promptfoo #12004) with the steps to reproduce it and the limits of what we showed.

Driftproof is an open-source tool that measures whether the instructions you give an AI assistant actually help, and keeps the evidence open for anyone to check. See how it works, or read the problems we've found in our own tool.