ProofRun

ProofRun is a local verification receipt for AI coding agents.

AI coding agents often say things like “tests pass,” “the build succeeded,” or “lint is clean.” The problem is that those results may have been produced against an earlier version of the code.

A test really can pass — and then become irrelevant three edits later.

ProofRun makes the claim itself checkable.

It runs the real command, records the real exit code, and binds that result to the exact Git state it ran against. Change the code afterward and a previous PASS automatically becomes STALE.

No LLM decides whether the code is correct. No agent gets to verify its own claim.

See it in one command

$ proofrun run test -- pytest
...
test: pass (exit 0, 1841ms)

$ proofrun status
test                 PASS    (exit 0, 1841ms)

Now change the code:

$ proofrun status
test                 STALE   (last run: pass, exit 0 — code changed since)

The previous test really did pass. It just no longer proves anything about the code you have now.

That distinction is what ProofRun exists to preserve.

Four explicit states — never a guess

Every check has one of four states:

• PASS — the check ran successfully against the current code
• FAIL — the check ran and returned a non-zero exit code
• STALE — the check ran previously, but the code has changed since
• NOT RUN — there is no valid result for the current code

There is no fifth state for “probably fine.”

ProofRun does not infer results from source code, previous conversations, or an AI agent's confidence.

Bound to the exact code state

A result is fingerprinted against:

• the current Git commit
• staged changes
• unstaged changes
• relevant untracked files

Change a single byte and ProofRun recomputes the fingerprint. If it no longer matches the state that produced the result, the check becomes STALE.

This makes it much harder for an AI coding workflow to accidentally reuse an old success as evidence for new code.

Tamper-evident local receipts

Since v0.3, every stored check result is signed with HMAC-SHA256 using a random machine-local key.

If someone simply edits receipt.json and changes a FAIL into a fake PASS, the signature no longer verifies and ProofRun refuses to trust that result.

The signing key is generated locally in .proofrun/secret. ProofRun makes a best-effort attempt to keep it out of Git, and if the key itself becomes Git-tracked, ProofRun refuses to trust it.

This guarantee is deliberately narrow:

• it is tamper-evident, not tamper-proof
• it protects against casual local receipt editing
• anyone who can obtain the local secret can forge a valid local signature
• it is machine-local evidence, not remote attestation
• it is not third-party-verifiable public-key proof
• it does not claim to prevent rollback or replay of an older genuinely signed receipt

ProofRun tries to make its trust boundary explicit rather than claiming more security than it actually provides.

Built for AI-assisted development

ProofRun is intentionally small and composable.

It can sit between an AI coding agent and the statement:

“Everything passes.”

Typical uses include:

• verifying an AI agent's test claims before accepting a change
• local development and pre-commit checks
• automated coding-agent workflows
• CI gates
• GitHub pull requests
• multi-agent development workflows where implementation and review are separated

ProofRun itself makes zero LLM calls.

It starts real subprocesses and observes their real exit codes.

Exact commands, not fuzzy shell strings

Checks are declared explicitly in .proofrun.yml:

checks:
  test:
    command: [pytest]
    required: true

  build:
    command: [npm, run, build]
    required: true

  lint:
    command: [ruff, check, .]
    required: false

Commands are represented as argument arrays rather than flattened shell strings.

That distinction matters: a command that only looks similar after shell quoting or string normalization should not be allowed to satisfy a different declared check.

GitHub Action

ProofRun also works as a GitHub Action:

on: pull_request

permissions:
  contents: read

jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - uses: yebiguo/proofrun@v1

The Action independently checks out the exact pull-request head, discards any local ProofRun receipt that came from the branch, and re-runs the configured checks itself.

That means the CI gate does not trust a receipt.json submitted by the same code change it is supposed to verify.

Built with AI agents — and reviewed by another one

ProofRun is itself an experiment in accountable AI-assisted software engineering.

Implementation is primarily performed by an AI coding agent under human direction. Changes then go through an independent, read-only adversarial review by a second agent before they are merged.

That second reviewer currently includes OpenAI Codex.

The review process has found real bugs rather than merely producing stylistic feedback.

Before the first release, independent review found several ways ProofRun's own command-matching logic could report a misleading PASS, including an ordinary shell-quoting mistake that caused a test command to silently run zero tests.

During development of v0.3, review found another trust-boundary problem: a symlinked .proofrun/ directory could cause an attacker-controlled signing key to be adopted as trustworthy. The issue was reproduced and fixed before release.

The same rule applies to ProofRun itself as to the agents it monitors:

a claim that something works is not enough — there should be evidence.

What ProofRun deliberately does not do

ProofRun is not:

• a code-quality judge
• an AI reviewer
• an autonomous coding agent
• an auto-fixer
• a test-output interpretation engine
• a remote attestation service
• a distributed trust system

Its job is intentionally narrower:

record what actually ran, bind it to the code it ran against, and make stale or tampered evidence visible.

Roadmap

Current development is focused on:

• structured output support for common test runners such as pytest, Jest, and JUnit
• stronger protection against a pull request weakening its own .proofrun.yml
• exploring stronger evidence models while keeping remote attestation and public-key trust separate from ProofRun's current local trust model

ProofRun is still young and pre-1.0, so the scope remains deliberately conservative.

Open source

ProofRun is written in Go and released under the MIT License.

Issues, technical feedback, and pull requests are welcome.

Trust the evidence, not the agent's claim.