Mitigating Tool Overload with Orchestrators and Dynamic Skills

9 min read

If overlapping tools are the problem, a different architecture might be the fix. Two designs are common. An orchestrator splits the work across specialist agents, each with a small tool set of its own. Dynamic skills, or on-demand tool loading, show the model only short group descriptions at first and load the full tool definitions when the model asks for them. Both show the model less at once, which is exactly why both are expected to help.

We built both on the Unique platform and ran them on the same tasks as the single agent from the sequential workflow study. Each one buys something real. Neither one removes the problem that four overlapping tools create.

arch-anatomy.png

Figure 1: Where the tool definitions live in each design. A thick line is a tool name, the thin lines under it are its description and parameters. (a) The single agent reads every definition in every request. (b) The orchestrator's coordinator reads one short summary per specialist, and each specialist reads only its own tools. (c) With dynamic skills (labelled on-demand loading) the agent starts from one line per group and loads a whole group with one tool_search call.

Three designs at a glance

Single agent

Idea: one agent, every tool definition in every request.

Buys: simplicity; no router can hide a needed tool.

Costs: the most tokens per task; document-heavy tasks can outgrow the context window.

Runs on: GPT-4o, GPT-5.1, GPT-5.5

Orchestrator

Idea: a coordinator hands each part of the task to one of five specialists, each owning one MCP server's tools.

Buys: room in the context window, because the coordinator never holds the documents.

Costs: about twice the model calls and 1.5 times the waiting time.

Runs on: GPT-4o, GPT-5.1, GPT-5.5

Dynamic skills

Idea: the agent sees seven group names and a tool_search tool, and loads a group when it needs it.

Buys: the same accuracy at about a fifth of the cost.

Costs: depends on the vendor's tool-search feature; its protection is only as good as the group descriptions.

Runs on: GPT-5.5 only

How we compared them

Twelve tasks were picked in advance from the sequential workflow study, using only the single agent's earlier results: tasks where an architecture change had room to make things better or worse. Seven of them could run under all three designs. Dynamic skills rely on the vendor's tool-search feature, which is only supported from GPT-5.4 onwards, so every three-way comparison on this page uses those seven tasks on GPT-5.5.

For the test with overlapping tools, each design got the same four decoys as in the earlier studies. The single agent sees them in its tool list. The orchestrator carries them directly on the coordinator, next to the specialist descriptions, as if an extra MCP server had been left enabled. With dynamic skills they form a group of their own behind a short group description, and we also ran a version with the four definitions permanently visible, so that hiding alone could not explain the result.

Why this is a price list and not a ranking

The orchestrator ran one trial per task where the other two ran three. Its coordinator only ever calls specialists, so our automatic per-task checks cannot see its tool calls, and its answers were scored by hand against the same task definitions. At seven tasks, none of the accuracy differences between the designs is statistically significant. What the comparison does show is how each design spent tokens, requests and time on these seven tasks, and which failures it does or does not prevent.

Every chart and number behind this page is in the public Architecture Evaluation findings report.

The price list

On the clean 110-tool set, all three designs completed all seven tasks. What they spent to get there differs a lot (averages per task on GPT-5.5, with the ratio to the single agent in brackets):

Single agent

Orchestrator

Dynamic skills

Tasks solved

7 of 7

7 of 7

7 of 7

Cost per task

$1.96

$1.23 (0.59×)

$0.44 (0.22×)

Input tokens per task

385,000

222,000 (0.54×)

79,000 (0.21×)

Largest single request

122,000

74,000 (0.57×)

37,000 (0.29×)

Model calls per task

5.3

12.1 (2.22×)

3.6 (0.66×)

Time waiting for the model

baseline

1.55×

1.02×

The largest single request decides whether a run fits in the model's context window at all; the input tokens per task are what you pay for. The two move independently, which is why we keep them apart. Costs are list prices and do not account for prompt caching. The ratios are the typical per-task ratio across the seven tasks, so they differ slightly from dividing the averages; the orchestrator's cost saving did not reach statistical significance, the dynamic-skills saving did.

The dynamic-skills saving comes from loading fewer definitions, not from cheaper ones: a loaded definition is billed in full. The benefit therefore depends on how well the groups split up the tool set, and it shrinks for a task that needs most of them.

When the context window runs out

The orchestrator's clear win is against the failure the sequential workflow study found on GPT-4o: runs whose conversation grew past the 128,000-token window and returned nothing. Three tasks were picked in advance to test exactly that. The orchestrator completed all four of those cases (three on GPT-4o, one on GPT-5.1); in three of them the single agent had returned nothing on every trial. The orchestrator's largest requests stayed at 60,000, 84,000 and 107,000 tokens, because the coordinator never holds the documents. It only receives the specialists' written summaries.

These are four single-trial cases, so they show that the mechanism works, not how often it would. The fourth case is also weaker: on GPT-5.1 the single agent had already answered correctly on one of its three trials, and there the orchestrator's largest request (222,000 tokens) was slightly larger than the single agent's (214,000), so holding less at once cannot explain that one.

That has a flip side. The coordinator cannot check a figure it never saw, and 60% of the orchestrator's output is written inside a specialist and then written again for the user. Dynamic skills were never tested on these cases, because those cases ran on GPT-4o and GPT-5.1, which do not support the tool-search feature.

Four decoys: no design escapes

With the four decoys added, accuracy dropped under every design, and with the decoys in view every design called one in most of its runs. How far it dropped depends on where the decoys sit, so we report both configurations.

We do not read a ranking into these numbers. The order changes with where the decoys sit, none of the differences is statistically significant at seven tasks, and the orchestrator's coordinator had far fewer entries to read than the single agent, which may have made it less distracted rather than better protected. What holds under every configuration is simpler: the designs change how many definitions the model reads, and when. None of them changes what a definition says.

Where the time goes

arch-sequence.png

Figure 2: The request traffic of one task under each design. Solid arrows are requests to the model, dashed arrows are returns; the bars show roughly how much each one carries.

On this platform, response time is driven by the number of model calls and the number of tokens written, much more than by prompt size. Each model call costs about 2.7 seconds before anything is written, each output token about 0.011 seconds, and each 1,000 input tokens only about 0.035 seconds. One extra model call therefore costs about as much time as 77,000 extra input tokens.

Two consequences follow. The orchestrator is slow because it more than doubles the number of arrows in the figure. Dynamic skills save about 306,000 input tokens per task, which is worth only around eleven seconds, and their model latency came out level with the single agent's (1.02×, not statistically significant). Token savings buy cost and context headroom, not speed.

What users noticed

detect-diagnose.png

Figure 3: Fourteen advisors used an orchestrator that carried the four decoys. Top: how far each of them got in spotting the problem. Bottom: the reason each one gave for the failure.

Fourteen practitioners worked through two advisory tasks with the single agent and the orchestrator, then repeated one task on an orchestrator that also carried the four decoys. That last workspace ran on an earlier model, so the failure is not pinned on the orchestrator. Every one of those 14 runs used a decoy. 10 of the 14 noticed on their own that something had gone wrong, but only 5 traced it to the agent taking the wrong route to the data, and 11 of the 14 still rated the answer at least partly correct. The speed difference did not register either: on the first task the orchestrator took about 1.7 times as long, yet participants split almost evenly on which workspace had felt faster.

With fourteen participants these are early signals, not settled results. They do point one way: overlapping tools have to be caught by whoever maintains the tool set, because most users in this study could not diagnose them.

Which design when

  • Fix overlapping tool descriptions first. No design removed the decoy failure, so the architecture choice comes second.

  • Consider the single agent when the tool set is distinct and tasks fit comfortably in the context window.

  • Consider dynamic skills when cost and prompt size matter, the model supports tool search and the tools split cleanly into groups. Write the group descriptions as carefully as the tools, because that is where the protection lives.

  • Consider an orchestrator when tasks pull in documents that overflow the context window, and budget for about twice the model calls and longer response times.

  • Whatever you choose, check that the figures are grounded, and log tool calls where they happen: with an orchestrator, inside the specialists.

What this page does not show

  • It is a price list, not a ranking. Seven shared tasks, one model for the three-way comparison, and a single-trial, hand-scored orchestrator cannot rank the designs on accuracy.

  • Dynamic skills were never tested on the context-overflow cases. Those ran on models that do not support the tool-search feature.

  • Nothing was tuned. No design received context or memory management beyond the platform defaults.

  • The numbers cover one setting. A banking tool set on one vendor's models; totals across designs describe these seven tasks, not typical costs.

Last updated