← All articles

Matching model and effort to each agent role, one setting at a time

The Guildwise HQ team · 8 November 2026 · 5 min read

• How-to
• AI agents
• LLM ops

Most agent teams start with one model for every role. That is quick to set up, but it hides two decisions that should be made separately: which model a role uses, and how much reasoning that model spends before it answers. This article sets out a way to make both choices per role, record them, and test them again when something changes.

Treat model and effort as two settings

The model sets the baseline capability and the price per token. The effort setting sets how much reasoning the model applies to each request. OpenAI's reasoning guide describes the execution mode and the effort setting as independent controls, and says effort falls back to medium when it is left out on GPT-5.6.

Effort is not free. Reasoning tokens are billed as output and count toward rate limits, so a higher effort setting raises both cost and latency on every call that uses it. That makes effort a cost lever you can set per role, not only a quality dial.

Step 1: Price each role by the cost of a wrong answer

Start with a plain question for each role: what does a wrong output cost, and who or what catches it before it ships? The NIST AI Risk Management Framework is voluntary guidance, and its Generative AI Profile proposes actions that match generative AI risks to an organisation's goals and priorities. You do not need its full process to apply the same logic to agent roles.

A simple way to sort roles:

• Errors that spread to every later step, such as a wrong plan, deserve the strongest model and a higher effort setting.
• Errors that a test suite or a second reviewer catches quickly can run on a mid-tier model at medium effort.
• Errors that are cheap to fix and easy to see, such as a status summary or a label, can run on a small model at low effort.

A public GitHub issue from one team shows the same split in practice. It assigns a faster model to implementation and triage, and keeps the stronger model on review at high effort, because the review step caught test-design weaknesses that the implementation step missed. Treat that as one team's trial, not a general result, and run your own trial before you copy it.

Step 2: Pin each role's settings, and check what the provider really applies

For each role, write down four values: the model, the effort level, the provider, and the date you last tested the pair. Keep them in the same configuration file as the role's prompt and tool list, so a reviewer can see them in one place.

Do not assume that a label means the same thing across providers. DeepSeek's documentation maps requested effort levels to different actual levels. Its table shows a request for medium is applied as high, and a request for minimal is applied as low. A team that copies a medium setting from one provider to another can get a noticeably heavier run without changing a single line of its own code.

Settings also change over time. Anthropic's documentation says that on Claude 4.7 and later models, a manual thinking budget returns an error, and that effort is the way to control the budget instead. A role configured with a fixed budget will fail after an upgrade. Check the provider's current reference before each model change, and treat a 400 error on a role as a configuration bug, not a retry case.

Step 3: Use routing only where a grader can judge the output

Model routing sends each request to a stronger or a weaker model based on a learned score. The RouteLLM paper reports that its routers cut costs by more than 2x in some cases on benchmarks without a drop in response quality. Those results come from question-answering style benchmarks, where the right answer is easy to grade.

Agent roles are harder. A planner's output is judged later, after the implementation and tests have run. A router that sees only the incoming request cannot tell whether a plan will hold up. For that reason, assign a fixed model and effort to each agent role, and use routing only inside a narrow, gradable class of work, such as sorting incoming tickets into labelled groups that a person can spot-check.

Step 4: Measure effort against cost, not only quality

A common mistake is to judge effort by the quality score alone. A recent arXiv preprint on evaluating agentic AI found that effort had a weak effect on the metric each agent was assigned to optimise. It argued that effort should be judged on how it changes cost and process as well as performance. Apply the same check to your roles: record tokens, run time and the number of accepted results, not just whether a run passed.

A simple test plan for each role:

1. Collect a small set of past tasks that the team already accepted or rejected.
2. Run the role at two effort levels with the same model, then run the cheapest model at the lower level.
3. Record accepted results, rework count, tokens and time for each run.
4. Keep the cheapest pair that meets the acceptance bar for that role. Move up only when a real failure shows the bar is not met.

Checklist before you change a role's model or effort

• The role's failure cost is written down, with the step that catches its errors.
• The model and effort are recorded with the provider and the test date.
• The label has been checked against the provider's current mapping, not assumed.
• No fixed thinking budget is set on a model that rejects it.
• Routing is used only for classes of work a person or test can grade.
• The test set includes cost and rework, not only the pass rate.
• A change to any role is run again on the same test set before it goes live.

Key takeaways

• Model and effort are separate settings. Assign both per role, and record them with the provider and test date.
• Sort roles by the cost of a wrong answer and by how quickly an error is caught downstream.
• Check what each provider actually applies for a given effort label, and treat rejected settings as configuration faults.
• Use routing only where a grader can judge the output. Measure effort against cost and rework, not only quality.

This article was written with AI assistance and researched from the sources listed.

Sources

1. Reasoning models
2. Adaptive thinking - Claude Platform Docs
3. Prompting best practices - Claude Platform Docs
4. Understanding the context window
5. Thinking Mode
6. RouteLLM: Learning to Route LLMs with Preference Data
7. AI Risk Management Framework
8. An Experimental Design Approach to Evaluating Agentic AI's Autonomous Model Discovery
9. Set model and effort per agent role: Sonnet for implementation, triage, orchestration and the loop; Opus high for review #5211

This article was written by an AI agent for Guildwise HQ (Guildwise blog writer, claude-haiku-5-5) and is reviewed against its sources.