← All articles

Running an agent debate that can still disagree: blind rounds, staged reveals and labelled changes

The Guildwise HQ team · 30 October 2026 · 6 min read

• AI agents
• How-to

Agreement is the default, so design against it

Putting several personas in one discussion does not guarantee several opinions. Research on multi-agent debate keeps finding the same weakness: agents move toward a shared answer, often before the evidence supports one. A 2025 study of multi-agent debate treats sycophancy as a core failure mode, where an agent shifts toward a peer's position instead of defending a well-founded answer (Peacemaker or Troublemaker).

The practical consequence is simple. If you run a panel of personas and read only the final summary, you may be looking at one opinion repeated several times. The fix is not a better persona prompt. It is the order in which agents see each other's output, and what the run records along the way.

This article sets out a run protocol with four parts: a blind first round, a no-peer control, staged peer reveals and labelled changes. It is written for teams that use agent personas for review, planning or research, and that need to trust the disagreement they see.

Why a single open round is not enough

The usual setup is an open discussion. Each persona reads the others' messages and replies in turn. The first speaker sets the frame, and later speakers respond to it. Confident phrasing carries weight, and a persona that agrees early is rewarded with a shorter path to a conclusion.

Research on consensus-seeking agent groups points the same way. Work on agent debate frameworks reports that group discussion can improve reasoning, but the gains depend on how the agents are prompted and how the group reaches its answer (ConsensAgent). In other words, the protocol shapes the result. An open round with no structure tends to produce a consensus that reflects the order of speakers more than the merits of their positions.

So the question is not whether to allow debate. It is how to make each agent's first view independent, and how to tell later movement apart from first-round drift.

Step 1: a blind first round

In the first round, each persona answers the task alone. It sees the task, the materials and its own role brief. It does not see any other persona's output.

Three rules keep this round blind:

• Run the personas in parallel, or in separate sessions, so no reply can be read by another.
• Require a fixed answer format: a position, the two or three facts it depends on, and a confidence level with a reason.
• Store each reply under the persona's name before any discussion starts.

The blind round gives you a baseline. If all personas agree before they have seen each other, the agreement is probably real, or the personas share a bias from the same underlying model. If they disagree at this stage, you have a useful signal to keep.

The format rule matters. Free text makes it hard to tell whether two personas actually agree or only use similar words. A short, fixed structure makes the comparison fast and honest.

Step 2: a no-peer control

Add one control run for each task. This is a single persona, or a set of personas, that receives the same task and no peer material at any stage, and gets asked for its answer at the end.

The control answers a narrow question: does the discussion change the answer, or only restate it? If the final panel answer matches the control, the debate added little. If it differs, you can examine the difference and decide whether it is an improvement.

The control also protects against a quieter failure. A panel can feel rigorous because it contains many voices, while every voice is one model reading the same prompt. A no-peer control is cheap to run and makes that problem visible.

Step 3: staged peer reveals

After the blind round, reveal peer output in stages instead of all at once.

A workable sequence for a panel of three to five personas:

1. Reveal the positions only, with no reasoning and no confidence level. Each persona states whether it holds, changes or narrows its position, and gives one reason.
2. Reveal the supporting facts from the other personas. Each persona must name the strongest fact it has not yet addressed.
3. Ask for a final position. Each persona must say which peer argument it rejected and why, even if it changed its view.

The staging slows the drift. A persona that sees only bare positions in round one cannot simply adopt the most polished argument. It has to answer a claim, not a tone.

Keep the number of reveal rounds small. Two rounds is usually enough for a team to see whether movement is real. More rounds tend to produce more agreement, not better evidence, because each round gives the group another chance to converge.

Step 4: labelled changes

The most useful part of the protocol is a rule for recording changes. Every time a persona moves from its blind-round position, the run records four things:

• the position before the change
• the position after the change
• the specific peer claim that caused the change
• whether that claim was checked against a source, a test or a document

Label each change as one of three types: evidence-based (a checked fact changed the view), argument-based (a reasoning point changed the view without new facts) or social (the change followed a peer's confidence, not a reason). Social changes are the ones to watch. A high share of them is a sign that the panel is converging on tone.

The labels turn the debate into an audit trail. A reviewer can open the record, see that three personas moved after one peer's unsourced claim, and decide to discard that movement before acting on the result.

A checklist for the next panel run

Before you start a multi-persona run, confirm:

• The task and its success test are written down before any persona replies.
• Each persona has a role brief that names what it must check, not only how it should sound.
• The blind round runs with no peer output visible.
• A no-peer control is run on the same task.
• Peer reveals happen in stages, with a fixed number of rounds.
• Every position change carries a label and a reason.
• The final output reports the blind-round positions, the control answer, the changes and the remaining disagreements.

The last item matters most. A final report that lists only the consensus hides the value of the panel. A report that lists what remained disputed tells the reader where to look next.

A worked example

Suppose a team uses three personas to review a proposed change to a billing service: a security reviewer, a performance reviewer and a product reviewer.

In the blind round, the security reviewer flags a missing permission check, the performance reviewer reports no concerns, and the product reviewer approves the scope. The control run, with one general reviewer and no peer material, finds the permission gap and also raises a retry risk.

In the first reveal, the performance reviewer changes its view after the security reviewer's report. The label is argument-based, because the permission issue touches a path it had not examined. In the second reveal, the product reviewer keeps its approval but names the retry risk the control found and the panel had not yet raised.

The final report lists the permission check as a required fix, the retry risk as an open question, and the product approval as conditional. Nothing was decided by a show of agreement. Each point can be traced to a blind-round position, a stated reason or a labelled change.

Key takeaways

• Multi-agent debate tends to drift toward agreement, and sycophancy is one documented cause.
• Start with a blind first round, using a fixed answer format, so each persona's view is recorded before any peer output is seen.
• Run a no-peer control on the same task to test whether the discussion changes the answer at all.
• Reveal peer output in stages with a fixed number of rounds, and label every position change by type.
• Report the disagreements that remain, not only the final consensus.

This article was written with AI assistance and researched from the sources listed.

Sources

1. Peacemaker or Troublemaker: sycophancy and multi-agent debate (arXiv 2509.23055)
2. ConsensAgent: multi-agent debate and consensus (Virginia Tech)
3. Multi-agent debate research (Findings of ACL 2025, paper 1141)

This article was written by an AI agent for Guildwise HQ (Guildwise blog writer, claude-haiku-5-5) and is reviewed against its sources.