AI agent orchestration: coordinating more than one agent

AI agent orchestration is how you coordinate multiple agents — or multiple steps of one agent — so the work completes, the failures surface, and the cost stays bounded. It matters because the naive approach, letting agents call each other freely, produces systems that are impossible to debug and expensive to run.

Start by asking whether you need more than one

The strongest advice here is deflationary: most problems solved with three agents are better solved with one agent and three tools.

Multiple agents are justified when you need genuinely different context windows — one agent shouldn’t carry the other’s entire history — or different permissions, where a research agent reads widely and a writing agent holds narrow write access. Or parallelism, where independent subtasks can run at once.

Multiple agents are not justified by wanting to name things after job titles. A “researcher”, a “writer” and an “editor” passing text between them is usually one agent with three prompts, plus three times the latency and three chances to lose the thread.

The patterns that hold up

Supervisor. One agent owns the goal and delegates subtasks to workers, each returning a result. The supervisor holds the plan; the workers hold nothing. This is the default for good reason: there’s exactly one place where state lives and one place to look when it goes wrong.

Pipeline. Fixed stages, each an agent, output feeding forward. Predictable and easy to reason about — but if the stages are fixed you should ask whether stages two and three needed to be agents at all.

Parallel fan-out. Same task, several workers, results merged. Genuinely valuable for independent work: reviewing a document against five criteria, or checking a claim from several angles. The merge step is where the design effort goes.

Debate or panel. Multiple agents produce competing answers, then a judge selects or synthesises. Expensive, and worth it only when being wrong is costlier than the extra inference.

The pattern that doesn’t

Free peer-to-peer delegation — any agent can call any other — looks elegant on a whiteboard and behaves badly in production.

You get cycles, where A asks B which asks A. You get runaway cost, because nothing owns the budget. You get untraceable behaviour, because the causal chain is spread across a dozen conversations with no single owner. And you get the compounding-error problem at its worst: each handoff is a chance to lose precision, and nobody notices until the output is subtly wrong.

If you take one thing from this: something must own the plan. Hierarchies debug; graphs don’t.

What orchestration actually has to provide

Independent of pattern, four things separate a working system from a demo.

A budget that’s enforced. Token and step ceilings per run, checked before each call rather than tallied afterwards. Without this a retry loop is an unbounded invoice.

Failure semantics you chose deliberately. When a worker fails, does the run abort, retry, skip, or degrade? Answer it per stage. The default — an exception nobody catches — abandons work halfway with side effects already applied.

A single trace. Every step, every tool call, every handoff, in one ordered log with the run id attached. If you can’t replay a run, you cannot debug it and you cannot audit it.

One approval surface. This is the one that gets missed. If three agents can each independently take irreversible actions, you have three places to reason about safety. Route every destructive step through one gate so the policy is stated once, and the human sees one decision rather than three.

Cost and latency compound

Two agents aren’t twice the cost of one — they’re often more, because context gets re-sent at every handoff and each agent re-establishes what it’s doing.

Latency compounds worse, because handoffs are serial by nature. A five-stage pipeline where each stage takes four seconds is a twenty-second wait, and users experience that as broken rather than thoughtful.

Both are arguments for the same discipline: parallelise where the work is genuinely independent, and collapse stages that don’t need to be separate agents.

How to decide

Reach for orchestration when subtasks need different permissions, different context, or can run simultaneously. Stay with one agent when the work is sequential and shares context — which is most of the time.

Then whatever you choose, make sure something owns the plan, the budget is enforced, the trace is single, and the approval gate is one.

Start with what an AI agent is, or see how tool selection works underneath.