Single-Agent Tool Overload in Sequential Workflows

8 min read

Picking the right first tool is only the start. A real advisory request needs several tool calls in a row, and each call depends on the one before it. So we followed one agent through whole tasks, from the first search to the final answer, and asked whether the pattern from first-tool selection still holds.

It does, and the overlap effect gets stronger. Growing the tool set from 22 to 110 distinct tools still changed nothing we could measure. Adding the same four decoy tools as before cut the share of correct, grounded answers by more than half.

One task, step by step

What the advisor asks

"A client emailed about Nvidia. Find the benchmark email with subject BENCH-2026-T2-NVDA-OPEN, open it, identify the ticker, then pull a compact current market snapshot for that ticker. Finish with the client ask and the market snapshot in three bullets."

What the agent has to do

  1. search_email finds the message by its subject.

  2. get_mail_message opens it, and the agent works out which ticker it is about.

  3. FactSet_GlobalPrices pulls the market snapshot for that ticker.

  4. The agent writes three bullets, and the market figures in them must match what step 3 returned.

Each step feeds the next. If step 2 reads the wrong company, step 3 fetches the wrong prices, and the answer is fluent, specific and wrong. That is why we score the final answer and not only the tool calls.

Why chains are fragile

agent-loop.png

Figure 1: The agent loop. The model either answers or asks for a tool call; the runtime runs the tool, appends the result to the conversation, and sends everything back to the model for the next step.

Two things follow from that loop. A wrong result does not disappear: it stays in the conversation and shapes every later step. And the conversation only grows: every tool result is sent again with every later request, until the model answers or the context window is full. The rest of this page is about those two consequences.

22 to 110 tools: still flat

Across whole tasks, growing the tool set from 22 to 110 distinct tools left grounded correctness where it was: 52.7% at 22 tools, 47.3% at 69 and 50.2% at 110, pooled across GPT-4o, GPT-5.1 and GPT-5.5. The middle tool set is the lowest of the three, so the numbers do not even move in one direction, and between 22 and 110 tools no model showed a statistically significant change. The one exception sits in between: at 69 tools GPT-4o lost 15 paired runs it had answered correctly at 22 tools and gained none, mostly through wrong tool selections on a handful of tasks. At 110 tools the paired difference to 22 is not significant, so we read it as a caution about GPT-4o rather than a trend with the number of tools.

Four decoys hurt more across a whole task

Then we made the same single change as in the first-tool study: the four broad, overlapping decoys join the 110-tool set.

lookalike-multistep.png

Figure 2: The same tasks run on 110 distinct tools (blue) and after four overlapping tools are added (orange). Left: tasks where the agent used the right tools. Right: tasks that ended with a correct answer whose figures match what the tools returned in that same run.

Pooled across the three models, grounded answers fall from 50.2% to 23.3%. GPT-5.1 falls from 42.7% to 13.4% and GPT-5.5 from 67.9% to 39.3%, down 29.3 and 28.6 points on paired runs. On a single decision the same four tools cost 6.0 to 26.9 percentage points; across a chain they cost 23.8 to 40.8 points of correct tool use, roughly one and a half to five times as much depending on the model. The two studies use different tasks and score different things, so the multiplier is indicative; only its direction holds on every model.

Exact numbers per model

Model

Right tools used: 110 → 114

Grounded answers: 110 → 114

Drop in grounded answers

GPT-4o

69.4% → 28.6%

32.7% → 24.5%

-8.2 pp (not significant)

GPT-5.1

47.6% → 7.3%

42.7% → 13.4%

-29.3 pp

GPT-5.5

67.9% → 44.0%

67.9% → 39.3%

-28.6 pp

Only runs that finished in both conditions are compared. GPT-4o's smaller drop comes with a caveat: many of its 110-tool runs ran out of context and returned nothing, so the runs left to compare are mostly the easier ones. Over all its scored runs, without pairing, GPT-4o drops 15.6 points.

One wrong turn costs the whole run

81% of steps picked the right tool when the step before them was right. 35% did when the step before them was wrong.

A single early mistake does not stay a single mistake: every step after it becomes more likely to go wrong too. The task groups agree. Long tasks that need 8 to 15 calls succeed 26% of the time, against 47% for tasks that need 3 to 5. The weakest group is short, though: tasks that first have to work out which company or client a request is about succeed only 22% of the time. What they share with the long tasks is the number of points where one call's output becomes the next call's input.

What fills the context window

llm-request.png

Figure 3: Two model requests in one run. The second request resends the whole tool set and every earlier message, including the tool result the runtime just appended.

Some runs never produced an answer, because the conversation grew larger than the model's context window. GPT-4o, which has the smallest window of the three, lost 78 of its 408 runs this way; the two models with far larger windows almost never did.

The tool list alone does not explain it. The failure happens at 22 tools as well, and on GPT-4o the 114-tool set, the largest in the design, fails exactly as often as the 22-tool set. What fills the window is what comes back from the calls: a tool definition is sent once per request, whereas a tool result is sent in that request and in every request after it, as the figure shows. A single document search can return about 53,000 tokens, roughly 41% of a 128,000-token window. These runs used the platform defaults, with no summarisation or trimming of earlier tool results.

Cheaper runs are not cheaper answers

With the decoys present, two of the models became cheaper per run and less accurate at the same time, because a run that gives up early is an inexpensive run. Charged per correct answer, the picture flips:

Model

Grounded answers: 110 → 114 tools

Cost per run: 110 → 114 tools

Cost per correct answer: 110 → 114 tools

GPT-4o

32.0% → 16.4%

$0.96 → $0.79

$3.00 → $4.83

GPT-5.1

43.4% → 13.3%

$0.44 → $0.41

$1.01 → $3.11

GPT-5.5

67.9% → 39.3%

$2.68 → $2.78

$3.94 → $7.07

Rates here are over all scored runs, not just paired ones. Costs are list prices per model and do not account for prompt caching, so read them as upper bounds. Any efficiency claim that is not measured per correct answer can be met by failing faster.

What we would tell a team hitting this

  1. Test whole tasks, not just the first call. Overlap that cost up to 27 points on a single tool choice halved the completed tasks across a chain.

  2. Fix overlapping tools before adding more. Between 22 and 110 distinct tools we saw no measurable change; four overlapping ones halved the grounded answers.

  3. Pay extra attention to the first steps. Identity lookups and other early calls strongly predict how the rest of the run goes.

  4. Budget for tool results. Retrieved documents are resent with every later request. Whether trimming or summarising them prevents the failure was not tested here.

  5. Require grounded answers. Check that the figures in an answer come from a tool result in that run.

  6. Compare cost per correct answer. Cost per run rewards agents that give up early.

How we ran it

The full task bank and a findings report with every chart and number behind this page are published as public artifacts: the Multi-Tool Task Completion task bank and the Multi-Tool Task Completion findings report.

The 34 tasks

The Multi-Tool Task Completion study uses 34 multi-step advisory tasks across Outlook, FactSet, InternalSearch and WebSearch. The mailbox and knowledge base hold fixed benchmark content; market and web data are live, which is why figures are checked against what the tool returned in that same run. The tasks fall into six groups:

Task group

Tasks

What it probes

Reference short chains

6

Short chains built on tools the first-tool study selected reliably

Known confusable short chains

9

Short chains that start from prompts which already flipped between right and wrong in the first-tool study

Source ambiguity

5

Requests where more than one source could plausibly hold the answer

Long-horizon advisory

6

Workflows that need 8 to 15 tool calls

Evidence sufficiency

4

Requests where the right move is to stop, or to say the data does not exist

Depth probes

4

Chains that must resolve an identity before any real lookup can start

Models, tool sets and runs

Every task ran on the same four tool sets as the first-tool study (22, 69, 110 and 114 tools), on GPT-4o, GPT-5.1 and GPT-5.5, three times each: 1,224 runs in total. Six tasks turned out to be answerable without real tool use, or could not be completed as set up, and are left out of the scoring. Runs that ended without an answer because the context window was full are reported separately and not counted as wrong answers.

How answers are scored

A run counts as a success only if its answer is grounded: it says what the task asked for, and the figures it relies on match what the tools returned in that same run. An answer that happens to be right about the world but quotes a number the agent never retrieved counts as a failure. Each task has its own fixed list of checks: the figure that must appear, the entity that must be identified, and the source it must have come from. We tried an LLM judge first and demoted it to a cross-check, because it gave different verdicts to runs whose records were identical.

Scoring the answer matters because tool calls and answers disagree in both directions. 217 runs called the right tools and still produced a wrong or ungrounded answer. And on tasks with an expected tool chain, 29% of successes came from runs that picked a wrong tool at some point and recovered.

What this page does not show

  • The absolute success rates are not production estimates. The task bank was deliberately seeded with tasks already known to be unstable, so it is harder than an average advisory day. The comparisons between tool sets are what the design supports.

  • The decoys are controlled artifacts. They are the same four realistic stressors as in the first-tool study, not accidental production tools.

  • The four decoys changed several things at once. They are broader than the tools they sit beside, longer, and some tell the model to prefer them. The study shows the size and direction of the effect, not which of these properties drives it.

  • No context management was tuned. Summarisation or trimming of tool results might change the context-window results.

  • The flat result is a bound, and it covers one setting. It holds for distinct tools between 22 and 110, in a banking tool set, on one vendor's models.

Last updated