Unknown should stay unknown

When a tool reports on what an AI agent did, it is summarising. The agent touched twelve files, ran the tests, and the tests passed. A summary has to choose words, and the easiest words are confident ones: the change is safe, this file handles billing, the tests cover the new behaviour. Each of these may be true. Often the tool does not know.

We think the most important property of a tool that reports on agent work is not how well it summarises. It is whether it keeps track of how it knows what it says.

Four kinds of statement

Statements about agent work come in different kinds, and they deserve different levels of trust.

• Observed. The agent ran a package install. The file billing.ts was modified. These are records of events.
• Established by experiment. Removing this hunk makes the selected test fail. This is a causal statement, relative to a specific evidence command, and anyone can rerun it.
• Inferred. This file probably handles billing, judging by its name and imports. Useful, and sometimes wrong.
• Unknown. There is no evidence either way.

A summary that blends these hands the reader a level of confidence nobody earned. “The tests cover the new behaviour” can mean that the tests ran and passed, which is an observation. It can mean that removing the new code made them fail, which is an experiment. Or it can mean that the test files have names similar to the source files, which is a guess.

Why rounding up is tempting

Tools are judged by how useful they feel, and “inconclusive” feels like failure. Text produced by language models is fluent by default, and fluency reads as confidence. And the sheer volume of agent output pushes everything towards compression: nobody wants three paragraphs of caveats for every change.

Each of these pressures is reasonable on its own. Together they produce tools that say more than they know, at the moment when people rely on them most.

What we do instead

Our tools carry the kind of a statement along with the statement.

In DiffWitness, runtime protection records what it sees as observed, and those observations never count as proof about a change. Proof results are witnessed, unwitnessed or inconclusive. A flaky or timed-out run never becomes a causal claim. When a search for the smallest working part of a patch runs out of budget, the certificate says the search was bounded rather than presenting the result as final.

In IdleProof, explanations separate what was observed in the project from what was inferred. When the evidence does not support a claim about what a file is for, the explanation says so instead of inventing a business role.

The cost is that the output is less smooth. It says “we don’t know” more often than a summary would. We think that is the point. The person reading it is responsible for the product, and they need to know where the knowledge ends.

A test for any tool

For any statement a tool makes about agent work, ask two questions. What would have to be different for this to be false? Did the tool check?

If nothing the tool checked could have made the statement false, it is an inference at best, and it should be labelled as one.

That rule is simple to state and surprisingly hard to follow. It is also the rule we try to hold our own tools to.