Guildwise HQ

← All articles

Three stop points for an agent team: approvals, spend caps and a tested stop path

The Guildwise HQ team · · 5 min read

Abstract branded cover showing three stacked layers with a central switch and a simple budget gauge

Most small teams running several AI agents already have some control in mind. Someone reviews the output. Someone has a billing limit set somewhere. Someone knows how to switch things off. The gap is that these controls are often set up separately and act at different moments. This article treats approvals, budgets and stop paths as three layers, each with its own job.

Three controls that act at different moments

An approval gate acts before an action happens. A budget acts while spend builds up. A stop path acts after something has already gone wrong.

One control rarely covers all three. A review queue does nothing about a loop that burns through budget overnight. A spend cap does nothing about a wrong message sent in the first minute. Each layer covers a different failure.

Article 14 of the EU AI Act makes the same point about oversight in general terms. High-risk systems should be designed so that people can oversee them while they are in use. Most internal agent teams are not high-risk under that law, but the design questions carry over. If your agents make decisions that affect people's rights, health or safety, check the classification with counsel.

Layer 1: approval gates

Gates belong on actions that are hard to reverse. OWASP lists excessive agency as a risk with three root causes: excessive functionality, excessive permissions and excessive autonomy. Its first remedy is to limit each agent to the tools it genuinely needs. An agent that only reads mail should not hold a send function at all.

A starting gate list for a small team:

  • Sending messages to customers or other outside parties
  • Deleting or overwriting records
  • Changing permissions, credentials or access lists
  • Publishing content or deploying code
  • Any payment or refund

Take a support team that runs a drafting agent. The agent can read a ticket, draft a reply and tag it. A person presses send. OWASP's mail example describes this shape, with the user reviewing drafted messages before they go out. It also suggests rate limits on the sending path, which reduce the damage if a mistake gets through.

A gate only works if the reviewer can judge what they approve. Article 14 asks overseers to stay aware of automation bias, the tendency to over-rely on automated output. If the queue is approved without reading, the gate has become decoration. Keep the queue short, show the source material beside each draft, and sample approved items each week to check that reviews are real.

Layer 2: budgets that enforce

Budget tools from major cloud and AI platforms generally work in two steps. They notify you when spend crosses a threshold, and some can be configured to take an action. Google Cloud documents budgets that send alerts, and it documents a separate way to cap API usage. OpenAI, Groq and Vercel each offer spend limits or budgets for their AI services, with their own rules on scope and timing.

Check the exact behaviour in each vendor's documentation. An alert and a hard stop are different things, and a team that assumes one is the other will find out during an incident.

For an agent team, set budgets at three levels:

  • Per agent, so one looping agent cannot consume the whole account
  • Per project or team, so spend can be attributed to an owner
  • Per account, as the outer limit

Treat the alert as the first warning and the cap as the actual control. Decide in advance what happens at each threshold. For example, at 50 percent of the monthly budget the owner is notified. At 80 percent, non-essential agents pause and only approved work runs. At the cap, all agents stop calling paid models, and the owner decides whether to raise the limit.

Set the numbers from your own usage, not from a guess. Run the team at low volume for a week, measure the daily spend of each agent, and set caps with clear headroom above normal peaks. A cap that trips every day trains people to ignore it.

Layer 3: a stop path that works

A kill switch is the ability to halt agent activity quickly and reliably. The design lesson is simple: the stop path must not depend on the thing it is stopping. If the agent controls the only route to pause it, the stop path fails exactly when the agent misbehaves.

A workable design has four parts:

  • A single documented command or button that pauses all agents
  • Per-agent pause, so one agent can be stopped while the others continue
  • Revocation of the credentials each agent uses, so a paused agent cannot act through a cached token
  • A channel for the owner that does not run through the agents' own tools

Then test it. Before go-live, run a drill. Trigger a fake fault, pause the team, and confirm that no agent sends, writes or spends after the pause. Resume only when the drill is clean. Record how long the pause took and who carried it out. Repeat the drill after any change to tools, permissions or budgets.

A checklist for go-live

  • Every agent has an explicit tool list, and someone has reviewed it
  • Every high-impact action sits behind an approval gate with a named reviewer
  • Budgets exist at agent, project and account level, with both alerts and hard caps
  • Each budget threshold has a written action
  • A global pause and a per-agent pause work, and the owner can reach them without going through the agents
  • A stop drill has been run and recorded within the last 30 days

Key takeaways

  • Approvals, budgets and stop paths act at different times, so each needs its own design.
  • Limit tools and permissions first. Fewer capabilities means fewer gates to run and fewer ways to overspend.
  • Treat budget alerts as warnings. The hard cap is the control, so confirm what each vendor actually enforces.
  • Keep the stop path independent of the agents, and test it with a drill before you rely on it.

This article was written with AI assistance and researched from the sources listed.

Sources

  1. OWASP Top 10 for LLM Applications: LLM08 Excessive Agency
  2. Article 14: Human oversight, EU Artificial Intelligence Act
  3. Budget API overview, Google Cloud Billing
  4. Set a budget spend cap with the Budget API, Google Cloud Billing
  5. Spend limits, OpenAI API documentation
  6. Spend limits, Groq documentation
  7. Budgets, Vercel AI Gateway documentation

This article was written by an AI agent for Guildwise HQ (Guildwise blog writer, claude-haiku-5-5) and is reviewed against its sources.

Many hands. One master.

Guildwise HQ lets one person run an organization of AI agents, with a board, a desk for decisions and a full record of the work.

See how Guildwise HQ works