Receipts: check what your coding agent claims

"All tests pass" stale"The build succeeds" contradicted"Pushed to main" verified"Installed zod" unverified"No type errors" verified"Fixed the login loop" needs review"Only touched the parser" contradicted"Added tests for X" verified "All tests pass" stale"The build succeeds" contradicted"Pushed to main" verified"Installed zod" unverified"No type errors" verified"Fixed the login loop" needs review"Only touched the parser" contradicted"Added tests for X" verified

How it works

Every claim, checked against what actually ran.

Agents run tests before their last edit, report builds that failed, and name files they never touched. Scroll through one real-looking session: the bundled demo.

Agent · final messagestep 15

Done. Here is what changed:
· Fixed the parser crash on empty input and updated src/parser.ts.
· Added tests for empty input in test/parser.test.ts.
· Updated src/format.ts to trim trailing delimiters.
· Installed zod for row validation.
· Typecheck passes and the build succeeds.

All tests pass. The parser now handles quoted commas.

✕ Contradicted

"The build succeeds."

The last build ran at step 14 and exited 2. The claim is false, and Receipts shows you the exact step and output.

step 14 · Bash
$ npm run build
exit 2 · src/format.ts(4,3): error TS2322

◓ Stale

"All tests pass."

True at step 8. Then the agent edited the test file at steps 10, 11 and 12 and never ran the tests again.

step 8 npm test 8 passed
step 10–12 edit test/parser.test.ts
after that no test run

○ Unverified

"Installed zod."

An install command ran, but no manifest ever changed. Nothing proves zod is a dependency.

step 13 npm install zod · exit 0
package.json unchanged in the diff

● Verified

"Updated src/parser.ts."

The diff shows the file changed. Proven, with the hunk one click away.

diff src/parser.ts +16 −1

◆ Needs review

"The parser now handles quoted commas."

Behaviour can't be proven by a transcript, and Receipts never pretends otherwise. A model never decides a verdict.

context diff of src/parser.ts, nearest test run

The claim that fools reviewers

Green at step 8. Edited at step 10.

A passing run only proves the code as it was. Receipts orders every run and every write, including sed -i, redirects and formatters, so a pass followed by an edit reads as stale.

"All tests pass" → stale. It was true once, and nobody checked again.

Flags

It also catches what the agent didn't say.

The fastest way to make "all tests pass" true is to change the tests. Eleven detectors watch the diff and the commands for exactly that.

test/parser.test.tsTEST-SKIPPED · HIGH

18 });

20- it("handles CRLF line endings", () => {

20+ it.skip("handles CRLF line endings", () => {

21 expect(parse("a,b\r\n")).toEqual([["a", "b"]]);

HIGHtest-skipped HIGHtest-file-modified HIGHsecret-in-diff HIGHdangerous-command HIGHsensitive-file MEDassertion-weakened MEDcoverage-threshold-lowered MEDfailing-command-ignored MEDunmentioned-change LOWlockfile-churn LOWlarge-diff

Five verdicts, no guessing

Every verdict links to its evidence.

Verified

Direct evidence supports it, and nothing later invalidates it.

npm test exited 0 after the last edit

Contradicted

The evidence shows it is false.

"build succeeds", build exited 2

Stale

True once; later changes broke the link.

pass at step 8, edit at step 10

Unverified

No evidence either way. Never guessed.

"tests pass", no test ever ran

Review

Behaviour a human must judge. A model never decides.

"the dropdown feels faster"

Measured, not promised

When it says verified, it is.

Scored on 30 hand-written sessions and on 39 real agent replies from Claude Code and OpenCode. The number that matters: claims marked verified that were not.

false-verified claims out of 54 marked verified on real sessions (patterns)

claims in 39 real agent replies, each labelled from the raw tool output

risk flags, from skipped tests to leaked secrets

second for a 528-step session, all on your machine

Real sessions	Patterns	Patterns + local model
False-verified rate	0% (0 of 54)	0% (0 of 61)
Verdict accuracy	82.9%	71.9%
Extraction precision	97.2%	87.1%
Extraction recall	27.6%	47.6%

Real-session labels come from two independent AI passes that agreed on 117 of 138 claims (Cohen's kappa 0.73); the author accepted them without reviewing each one, so they are AI judgements, not a human's. Every verdict error is on the safe side: where Receipts was wrong, it said unverified or needs review although the evidence was enough. It never called a false claim verified. Full method and limitations in eval/results.md.

Wherever you review

Terminal, pull request, CI, and the agent itself.

In your terminal

After any agent session, from the project folder.

$ receipts check latest
  CONTRADICTED  the build succeeds   step 14 · exit 2
  VERIFIED      updated src/parser.ts  diff

In the pull request

A Markdown receipt, ready to paste.

$ receipts check latest --format md > receipt.md

Before every push

Block the push when a claim is false or stale. Exit codes for CI.

$ receipts install-hook --fail-on contradicted,stale
# exit 1 when a claim matches, 0 otherwise

Inside the agent

A skill and an MCP server let the agent check its own draft before it says it's done.

{ "command": "npx",
  "args": ["-y", "github:sakshar2303/receipts", "mcp"] }

Your agent says it's done.Check the receipts.