← All articles

The exit report: making an agent say what it did not finish

The Guildwise HQ team · 20 October 2026 · 6 min read

• AI agents
• How-to

A done check tells an agent what success looks like. It does not tell you what the agent actually did, which checks it ran, or which parts it left open. That gap is where review time disappears. A reviewer receives a draft or a diff and then has to rebuild the run to learn whether the agent tested, skipped or guessed.

This post covers a short exit report that the agent returns with every result. It sits after the done check. The done check defines the target. The exit report records how far the run got against that target.

Why a good done check is not enough

A done check written before the prompt narrows the job. It still leaves one question open at review time: what happened during the run. An agent can produce a plausible result while skipping a check, using a stale input or quietly dropping a case.

Anthropic's guidance on agents points in a practical direction. The team reports that the most successful implementations use simple, composable patterns rather than complex frameworks. Simple patterns have clear handoff points, and those handoff points are where a reviewer needs information. The exit report puts that information in one place.

OpenAI's agent documentation describes a trace as the end-to-end record of model calls, tool calls, guardrails and handoffs for one run. A trace is good evidence, but a reviewer should not need to read a trace to decide whether to accept a result. The exit report summarises what the trace would show, and the trace stays available as proof.

The three parts of an exit report

Keep the report to three parts, in this order.

First, the result. Name the single artifact the agent produced, such as a file, a pull request or a draft. If the agent produced more than one artifact, the job is probably too broad for one agent.

Second, the check ledger. Copy each item from the done check into the report and mark it passed, failed or not run. Attach the evidence for each passed item, such as the output of a count comparison or the location of a test run. An item with no evidence is recorded as not run.

Third, the open items. List what the agent did not do, why it stopped, and what it would need to finish. Open items are the questions a human answers before acceptance.

A worked example: matching invoices to payments

Suppose the narrow job is this: match the week's invoices to the payment export and produce a list of mismatches. The done check has four items. Every invoice appears exactly once in the output. The totals in the output equal the totals in the export. Duplicate payments are flagged. Currency conversions match the rate table.

An exit report for that run might read like this.

Result: one mismatch list file.

Check ledger: every invoice appears exactly once, passed, with the row count compared to the source. Totals match the export, passed, with the difference shown as zero. Duplicate payments flagged, passed, with the flagged rows listed. Currency conversions checked against the rate table, not run, because the rate table for the last two days was missing from the input folder.

Open items: three invoices have no payment record. The agent left them unmatched and did not guess a payer. A finance person needs to confirm whether the bank feed for those dates was complete. The currency check cannot run until the rate table is supplied.

A reviewer reading that report knows the result cannot be accepted yet. The reason is stated, and the next action is clear. Without the report, the same run would look finished because the mismatch list exists.

Treat 'not run' as a pause signal

OpenAI's guidance on guardrails and human review frames the two controls together. Automatic guardrails validate inputs, outputs or tool behaviour. Human review pauses the run so a person or policy can approve or reject a sensitive action. Together they decide whether a run continues, pauses or stops.

The exit report brings that logic to the reviewer. A passed item lets the run continue toward acceptance. A failed item stops acceptance. A not run item pauses the decision until someone supplies the missing input or accepts the gap in writing.

The same guidance on planning for human intervention suggests setting limits on retries and actions, and escalating when the agent exceeds them. Record that stop reason in the report. A line such as 'stopped after three failed attempts to parse the export' tells the reviewer that the agent did not simply give up without reason.

Use trace-style questions as ledger rows

OpenAI's evaluation guidance lists questions that trace grading can answer. Did the agent pick the right tool? Did a handoff happen when it should have? Did the workflow violate an instruction or safety policy? Did a change improve the end-to-end behaviour?

These questions make useful ledger rows when they fit the job. For a support-reply agent, a row might read: no reply was sent without the policy check. For a code agent, a row might read: the agent did not change files outside the assigned folder. Each row needs evidence, and the evidence should be something a reviewer can open in a minute.

A checklist for the exit report

Add these rules to the agent's instructions, and check that every result follows them.

• Name the single artifact produced.
• Copy each done-check item into the ledger and mark it passed, failed or not run.
• Attach evidence to every passed item, or mark it not run.
• List open items with the reason and the input needed to finish.
• State the stop reason if a retry or action limit was reached.
• Do not mark the job complete while any item is failed, not run or open.

The last rule matters most. An agent that is allowed to mark its own job complete will do so under pressure. The report should show completion as a conclusion drawn from the ledger, not as a claim the agent makes on its own.

What to measure

The NIST AI Risk Management Framework organises risk work into four functions: Govern, Map, Measure and Manage. The exit report fits into Measure and Manage. It produces data on what the agent claimed against what the reviewer found.

Track two numbers for each agent role over a few weeks. The first is the reviewer time to accept or reject a result. The second is how often an item marked passed later turns out to be wrong, or how often a not run item later turns out to be a failure. The first shows whether the report saves time. The second shows whether the agent's claims can be trusted.

If the second number is high, tighten the evidence rule before adding more autonomy. If the first number is high because of many open items, the job may be too broad, and the narrowing exercise should start again.

Key takeaways

• A done check defines success. An exit report records how far a run got against it.
• Structure the report as result, check ledger and open items.
• Treat a not run item as a pause signal, not a pass.
• Record stop reasons when a retry or action limit was reached.
• Track reviewer time and later-found errors to see whether the report helps.

This article was written with AI assistance and researched from the sources listed.

Sources

1. Building Effective AI Agents
2. A practical guide to building agents
3. A Practical Guide to Building AI Agents
4. Guardrails and human review
5. Evaluate agent workflows
6. AI Risk Management Framework

This article was written by an AI agent for Guildwise HQ (Guildwise blog writer, claude-haiku-5-5) and is reviewed against its sources.