The agent finished, the output is wrong, and you have a transcript. How to find the first wrong step rather than debugging its consequences — and the six failure signatures visible in a trace.
- Reading the reasoning first is the common mistake: the reasoning is always plausible, because plausibility is what these systems produce.
When an AI does a multi-step job badly, you get a log of what it did. Do not start at the wrong answer — start at the first wrong step.
Everything after that point is just a consequence. And read the list of actions before you read its explanations, because the explanations always sound reasonable.
The agent finished. The output is wrong. You have a transcript. Reading it is a skill, and almost nobody writes about it.
Find the first wrong step, not the wrong output. Everything after the first error is downstream of it — debugging the final result means debugging a consequence.
Four questions, in order
Did it understand the objective?
Read the agent's first substantive action, not its restatement of the task. Restating is cheap; the first action reveals what it actually understood.
If the first action is already in the wrong place, the brief failed — not the agent. Stop here and fix the objective.
Where does the trace stop matching what you expected?
Scan the step list without reading the reasoning. You are looking for the first step you would not have taken. That is the turn.
Reading the reasoning first is the common mistake: the reasoning is always plausible, because plausibility is what these systems produce.
What did it read just before that?
Almost every wrong turn follows an input. A file with unexpected content, a search result, an error message, a comment. Look at the last thing it read before it changed direction.
If that input contained anything resembling an instruction, you have found an injection — see guardrails.
Did it report what it actually did?
Compare the summary against the step list. A gap between them is the signature of silent failure — and it is the most important thing a trace can tell you, because it means the summary cannot be trusted on any run.
Failure signatures in a trace
- Repeated near-identical steps → a loop. Something it needs is missing and it cannot recognise that.
- A long run that produced little → no stopping condition.
- Steps outside the named scope → the boundary was advisory rather than enforced.
- Confident summary, thin step list → silent failure.
- A direction change with no preceding input → the objective was ambiguous and it resolved the ambiguity itself.
- An input containing imperative text → injection.
What to change, given what you found
The fix belongs in the brief, not in a follow-up message. A correction typed mid-run fixes one run; a corrected brief fixes every future one.
If you cannot say which line of the brief would have prevented it, you have not finished diagnosing.
Most diagnoses end in one of four edits: a tighter scope, an explicit stopping condition, a required artefact, or the line forbidding action on fetched content. The brief template has all four.