Client-Ready Decks with GPT-5.1: Evaluation
15 min read
Accuracy vs time and cost
Standard skill and optimised setup on the 8 new-template requests, with GPT-6.1 Sol and Claude Opus 5.5 as reference points. The intermediate tools-and-rules step is in the Codex and pi tables under Results. Colour is the harness, shape the reasoning effort; whiskers are 95% intervals. pi numbers are experimental. The pi optimised setup has not run yet; it will be added to both charts once ready.
The optimised setup is what makes GPT-5.1 usable. It takes GPT-5.1 high from 26% to 72% client-ready slides on new templates, at about four times the time (5 to 20 minutes) and under three times the cost (USD 0.46 to 1.25).
GPT-6.1 Sol beats the optimised GPT-5.1 setup on accuracy, time and cost at once, at high and at medium. Claude Opus 5.5 is as accurate and as fast, but the most expensive.
pi runs GPT-5.1 with the standard skill today: the fastest and cheapest point, at the level of the Codex standard skill (28% against 26%).
Example slides on the Unique template, optimised setup next to the standard skill: Client-Ready Decks with GPT-5.1: Example Slides.
Key facts
What we test: how well Unique AI Conduct builds client-ready PowerPoint decks in a bank’s own template with GPT-5.1, the strongest model Microsoft offers with Swiss-only inference. 13 requests: 8 on 4 company templates nobody tuned the setup on (new templates) and 5 on 4 templates the setup was developed on (tuned templates). 248 decks, graded blind slide by slide (runs of 23 to 26 October 2026).
How we test: each request runs end to end through Conduct, as a relationship manager (RM) would send it. A review skill grades every deck blind against a 15-item checklist. The score is client-ready slides: the share of the requested slides that need no fix before the deck goes to the client.
Recommended setup: GPT-5.1 high on the Codex harness with the optimised setup. On new templates 72% [68–77%] of slides are client-ready, on tuned templates 97%. With the standard PowerPoint skill alone, GPT-5.1 high reaches 26% [20–33%] on new templates.
Reference models: Claude Opus 5.5 (Claude Code harness) and GPT-6.1 Sol (Codex harness) reach 99% [98–100%] and 98% [96–100%] on new templates with the standard skill, in under 5 minutes. Whether a client may use them with client-identifying data (CID) depends on the client.
pi harness: on master, GPT-5.1 on pi gets none of our GPT-5.1 setup and reaches 28% [20–36%] on new templates. With our build tools and working rules (experimental, not merged, no output checks) it reaches 65% [54–75%], against 72% [68–76%] for the same tools and rules on Codex. The intervals overlap on new templates; on tuned templates Codex is ahead (95% [93–97%] against 90% [87–93%]).
Time and cost: optimised setup at high takes about 20 minutes and USD 1.25 per deck on a new template, and 12.5 minutes and USD 0.90 on a tuned one. A deck is a task the RM starts and comes back to.
Caveat: the intervals measure run-to-run noise only (decks resampled within each request), not the choice of the 8 new templates. Medium ran one deck per request and has no interval. One grader, written by the team that built the setup.
Why this matters for relationship managers
An RM spends a large part of their client work producing documents: meeting preparation, portfolio reviews, proposals and pitch books. Most of them are PowerPoint decks in the bank’s corporate template, full of client names, holdings and family details. A deck that took an afternoon should take minutes, and it must look like the bank made it.
This material contains CID by nature. Some clients’ data protection guidelines require that inference on CID happens only in Switzerland, and GPT-5.1 is the strongest model Microsoft offers with Swiss-only inference. Frontier models such as Claude Opus 5.5 or GPT-6.1 Sol build good decks with a standard skill; which models a client may use on CID depends on the client. Where only Swiss inference is allowed, the agent has to work with a model that is noticeably weaker at long, multi-step, visually demanding tasks.
Our approach moves the hard parts out of the model. GPT-5.1 decides what goes on each slide; deterministic tools read the template, place the content in the template’s own slides and check the result before the deck is handed over.
What the RM gets:
The bank’s template, chosen and confirmed. The RM picks one of the bank’s templates or attaches their own, and every deck is built from it.
Real template slides. Content goes into copies of the template’s own slides: cover, agenda, KPI tiles, charts, tables, timelines, process flows, team slides. Fonts, colours, logos and footers stay the bank’s.
Charts from the client’s numbers that stay editable in PowerPoint.
A checked deck. Before the turn ends, output checks confirm that the template was selected, every figure the RM gave is on a slide, no template sample text or placeholder is left, and the model has looked at the rendered slides.
Languages. Tested in English, German and French.
What we test
Requests and templates
Set | Requests | What it measures |
|---|---|---|
New templates | 8 requests on 4 company templates (two banks, an asset manager, a private bank) in English, German and French | How the setup does on a bank’s own template it has never seen. Nobody tuned on these templates. |
Tuned templates | 5 requests on 4 templates in English, German and French | Templates the setup was developed on: what a bank gets once its template has been reviewed with us. |
The two sets are reported separately and never pooled: the tuned templates show fit, the new templates show how well the setup generalises. All figures in the requests are fictitious; no real CID is involved.
Grading
Every deck is graded blind, slide by slide, by the pptx-deck-review skill with a 15-item checklist: template sample content left, empty or half-filled parts, overlap, text out of its frame, requested content missing, off-template styling, a file that does not open, figures in the wrong place, leftover placeholders, unreadable text and wrong slide count. A finding the RM must fix before sending is a major finding.
Client-ready slides (main score): requested slides with no major finding, as a share of the requested slides. A deck that does not open counts as 0. Higher is better.
Unusable decks: a deck that does not open, has the wrong slide count, or has fewer than half of its slides client-ready. Lower is better.
Harnesses and setups
Harness | Models | Setups tested |
|---|---|---|
Codex | GPT-5.1 high and medium; GPT-6.1 Sol medium (reference) | Standard skill; tools and rules; optimised setup (on master) |
pi | GPT-5.1 high | Standard skill (on master); tools and rules (experimental, local configuration) |
Claude Code | Claude Opus 5.5 medium (reference) | Standard skill |
Standard skill: the shared PowerPoint skill, without anything specific to GPT-5.1.
Tools and rules: adds our GPT-5.1 build tools (template reading, slide placement, fit check) and working rules.
Optimised setup: adds the output checks that run before the turn ends. The checks exist for Codex only, so pi has no “optimised setup” step.
What to expect
Per deck on a new template, the bank’s own template before it has been reviewed with us (95% intervals in brackets):
Per deck, new template | GPT-5.1 high, Codex, optimised setup | GPT-5.1 high, pi, tools and rules (experimental) | GPT-6.1 Sol medium, Codex (reference) | Claude Opus 5.5 medium, Claude Code (reference) |
|---|---|---|---|---|
Client-ready slides | 72% [68–77%] | 65% [54–75%] | 98% [96–100%] | 99% [98–100%] |
Unusable decks | 8% [0–13%] | 21% [8–33%] | 0% | 0% |
Minutes per deck | 19.6 | 6.4 | 4.7 | 4.6 |
Minutes per requested slide | 3.69 [3.43–3.97] | 1.20 [0.98–1.42] | 0.91 [0.87–0.95] | 0.86 [0.84–0.89] |
Minutes per client-ready slide | 4.99 [4.48–5.62] | 1.88 [1.61–2.18] | 0.88 [0.83–0.93] | 0.86 [0.82–0.90] |
Est. cost per deck | USD 1.25 | USD 0.49 | USD 0.54 | USD 1.60 |
On a new template, about one slide in four still needs a fix with GPT-5.1. Once the template has been reviewed with us (tuned templates), 97% of slides are client-ready at high and at medium, level with Claude Opus 5.5.
Results
Client-ready slides and unusable decks: mean over each request’s decks, then over requests, so each request weighs the same. 95% bootstrap interval in brackets where the run has one. Minutes are wall-clock per deck; cost is the estimated model cost per deck at list prices.
Codex harness
New templates (8 requests; GPT-5.1 high 3 rounds, GPT-5.1 medium 1 round, GPT-6.1 Sol 2 rounds):
Model and setup | Reasoning effort | Client-ready slides | Unusable decks | Minutes per deck | Est. cost per deck |
|---|---|---|---|---|---|
GPT-5.1, standard skill | high | 26% [20–33%] | 71% [58–83%] | 5.3 | USD 0.46 |
GPT-5.1, tools and rules | high | 72% [68–76%] | 8% [0–17%] | 10.2 | USD 0.89 |
GPT-5.1, optimised setup | high | 72% [68–77%] | 8% [0–13%] | 19.6 | USD 1.25 |
GPT-5.1, standard skill | medium | 39%* | 62%* | 2.7 | USD 0.18 |
GPT-5.1, tools and rules | medium | 70%* | 0%* | 6.4 | USD 0.42 |
GPT-5.1, optimised setup | medium | 63%* | 12%* | 13.3 | USD 0.83 |
GPT-6.1 Sol, standard skill (reference) | medium | 98% [96–100%] | 0% | 4.7 | USD 0.54 |
Tuned templates (5 requests; GPT-5.1 high 2 rounds, GPT-5.1 medium and GPT-6.1 Sol 1 round):
Model and setup | Reasoning effort | Client-ready slides | Unusable decks | Minutes per deck | Est. cost per deck |
|---|---|---|---|---|---|
GPT-5.1, standard skill | high | 40% [24–57%] | 60% [40–80%] | 10.3 | USD 0.92 |
GPT-5.1, tools and rules | high | 95% [93–97%] | 0% | 9.8 | USD 0.87 |
GPT-5.1, optimised setup | high | 97%** | 0% | 12.5 | USD 0.90 |
GPT-5.1, standard skill | medium | 18%* | 80%* | 6.1 | USD 0.43 |
GPT-5.1, tools and rules | medium | 57%* | 40%* | 5.6 | USD 0.32 |
GPT-5.1, optimised setup | medium | 97%* | 0%* | 9.1 | USD 0.63 |
GPT-6.1 Sol, standard skill (reference) | medium | 84%* | 20%* | 6.4 | USD 0.78 |
* One deck per request, so no interval; a difference of one or two decks is within noise. ** Two rounds, but the benchmark report gives no interval for this value.
pi harness
GPT-5.1 high only, same requests and rounds as GPT-5.1 high on Codex. On master, pi gets none of the GPT-5.1 setup (the standard-skill row is what pi runs today). The tools-and-rules row is experimental and not merged: it ran through a local configuration, without output checks, because the checks have no pi version yet. We do not recommend pi for PowerPoint with GPT-5.1 today.
Set | Setup | Client-ready slides | Unusable decks | Minutes per deck | Est. cost per deck |
|---|---|---|---|---|---|
New templates | Standard skill (master) | 28% [20–36%] | 79% [71–88%] | 2.2 | USD 0.17 |
New templates | Tools and rules (experimental) | 65% [54–75%] | 21% [8–33%] | 6.4 | USD 0.49 |
Tuned templates | Standard skill (master) | 31% [21–40%] | 70% [60–80%] | 4.6 | USD 0.36 |
Tuned templates | Tools and rules (experimental) | 90% [87–93%] | 0% | 5.5 | USD 0.46 |
pi against Codex, same tools and rules, GPT-5.1 high:
Measure | Codex | pi (experimental) |
|---|---|---|
Client-ready slides, new templates | 72% [68–76%] | 65% [54–75%] |
Client-ready slides, tuned templates | 95% [93–97%] | 90% [87–93%] |
Unusable decks, new templates | 8% [0–17%] | 21% [8–33%] |
Minutes per deck, new templates | 10.2 | 6.4 |
Est. cost per deck, new templates | USD 0.89 | USD 0.49 |
Our tools and rules help on pi too (28% to 65% on new templates, against 26% to 72% on Codex). pi is faster and cheaper, but produces more unusable decks, and its spread on new templates is wide. Shipping the GPT-5.1 setup for pi needs the working rules in pi’s instructions, skill paths that work on pi and a pi version of the output checks.
Claude Code harness (reference)
Set | Model and setup | Reasoning effort | Client-ready slides | Unusable decks | Minutes per deck | Est. cost per deck |
|---|---|---|---|---|---|---|
New templates (2 rounds) | Claude Opus 5.5, standard skill | medium | 99% [98–100%] | 0% | 4.6 | USD 1.60 |
Tuned templates (1 round) | Claude Opus 5.5, standard skill | medium | 97%* | 0%* | 6.0 | USD 1.83 |
* One deck per request, so no interval.
Minutes per slide
Two time measures per setup, with new and tuned templates kept apart. Minutes per requested slide: wall-clock minutes of a deck divided by the number of slides its request asked for. Minutes per client-ready slide: minutes per deck divided by client-ready slides per deck, so the time spent on slides that still need a fix is charged to the ones that do not. 95% bootstrap interval in brackets where the setup ran more than one deck per request.
New templates (8 requests, 5.4 requested slides on average):
Model, harness and setup | Reasoning effort | Minutes per requested slide | Minutes per client-ready slide |
|---|---|---|---|
GPT-5.1 on Codex, standard skill | high | 1.02 [0.82–1.27] | 3.72 [2.89–4.75] |
GPT-5.1 on Codex, tools and rules | high | 1.87 [1.68–2.04] | 2.61 [2.40–2.80] |
GPT-5.1 on Codex, optimised setup | high | 3.69 [3.43–3.97] | 4.99 [4.48–5.62] |
GPT-5.1 on Codex, standard skill | medium | 0.52* | 1.21* |
GPT-5.1 on Codex, tools and rules | medium | 1.20* | 1.77* |
GPT-5.1 on Codex, optimised setup | medium | 2.50* | 3.68* |
GPT-5.1 on pi, standard skill (experimental) | high | 0.43 [0.35–0.52] | 1.51 [1.01–2.41] |
GPT-5.1 on pi, tools and rules (experimental) | high | 1.20 [0.98–1.42] | 1.88 [1.61–2.18] |
GPT-6.1 Sol on Codex, standard skill (reference) | medium | 0.91 [0.87–0.95] | 0.88 [0.83–0.93] |
Claude Opus 5.5 on Claude Code, standard skill (reference) | medium | 0.86 [0.84–0.89] | 0.86 [0.82–0.90] |
Tuned templates (5 requests, 6.4 requested slides on average):
Model, harness and setup | Reasoning effort | Minutes per requested slide | Minutes per client-ready slide |
|---|---|---|---|
GPT-5.1 on Codex, standard skill | high | 1.74 [1.44–2.05] | 3.95 [2.98–6.59] |
GPT-5.1 on Codex, tools and rules | high | 1.57 [1.34–1.80] | 1.61 [1.34–1.89] |
GPT-5.1 on Codex, optimised setup | high | 2.03 [1.83–2.22] | 2.02 [1.81–2.23] |
GPT-5.1 on Codex, standard skill | medium | 1.03* | 6.13* |
GPT-5.1 on Codex, tools and rules | medium | 0.84* | 1.46* |
GPT-5.1 on Codex, optimised setup | medium | 1.45* | 1.46* |
GPT-5.1 on pi, standard skill (experimental) | high | 0.70 [0.54–0.85] | 2.32 [2.08–2.86] |
GPT-5.1 on pi, tools and rules (experimental) | high | 0.89 [0.84–0.93] | 0.95 [0.92–0.98] |
GPT-6.1 Sol on Codex, standard skill (reference) | medium | 1.02* | 1.18* |
Claude Opus 5.5 on Claude Code, standard skill (reference) | medium | 0.96* | 0.96* |
* One deck per request, so no interval. pi numbers are experimental.
What the output checks add (Codex)
GPT-5.1 on Codex | Without checks (tools and rules) | With checks (optimised setup) |
|---|---|---|
High, client-ready slides, new templates | 72% [68–76%] | 72% [68–77%] |
High, decks missing a figure the RM gave (both sets) | 4 of 34 | 0 of 34 |
High, decks with a mechanical defect (both sets) | 9 of 34 | 5 of 34 |
High, minutes per deck, new templates | 10.2 | 19.6 |
Medium, client-ready slides, tuned templates | 57%* | 97%* |
Medium, unusable decks, tuned templates | 40%* | 0%* |
Medium, client-ready slides, new templates | 70%* | 63%* |
At high the average does not move, but the checks remove defects the RM would otherwise send out. At medium they are what lifts the tuned templates to the high result: without them, medium put every slide on the cover layout in one deck and typed content into small label frames in another. On new templates the template-selection check made medium swap the RM’s attached template for a Knowledge Base one in two decks. * One deck per request.
Which effort we recommend
High. On new templates it is ahead of medium (72% [68–77%] against 63%, one round) and medium switched to the wrong template twice. On tuned templates both reach 97%, and medium is about a quarter faster and a third cheaper there (9.1 against 12.5 minutes, USD 0.63 against 0.90), so medium is an option for a bank whose template has been reviewed with us.
Limitations
How flexible it is with new layouts
The gap between tuned and new templates is the honest measure of flexibility. On tuned templates GPT-5.1 matches Claude Opus 5.5; on a template it has never seen, about one slide in four still needs a fix. The reference models do not show this gap. Most of the remaining defects come from reading an unfamiliar template, not from the content the model writes:
Cover layouts the tools do not recognise, for example a cover in another language, or a cover layout without a text frame, so the deck gets a plain title slide or an untitled cover.
Guideline pages offered as content slides. Some templates contain pages of example shapes and usage notes; the tools can mistake them for real slide designs.
Fixed layout elements. Boxes drawn by the layout itself cannot be removed, so a design for five teams shows five boxes when the RM asked for three.
Template sample content left behind and design parts left empty when the request has fewer items than the design.
Template choice. In two medium decks, the template-selection step picked a Knowledge Base template instead of the one the RM attached.
What is being fixed
Next sprint: better reading of cover and guideline layouts, a safe way to remove slides so a deck always opens, and a delivery check that the file opens.
Fewer false alarms from the checks, which sometimes block a correct slide (for example an e-mail address the RM asked for) and push the model off a good design.
A compose step. An earlier compiler approach, where GPT-5.1 only writes typed content and a compiler builds every slide, made layout safe on the template it was built for but was not flexible enough across templates. We are folding its strongest part into the current tools: composing a slide from template parts when no template slide fits the content.
What the RM still has to do
Review the deck before it goes to the client. Content is checked by the file hallucination check (beta): a generated deck is checked against the sources the agent used, and the file card warns when it contradicts them. See Hallucination Check.
Fix about one slide in four on a new template, usually a cover, a leftover sample element or an empty design part.
Review each new template once with us before it is rolled out to RMs. The review catches template-reading issues before RMs meet them.
Diagrams the template does not contain (SWOT grid, BCG matrix) come as structured text, not drawings.
Next steps
Intervals over requests: resample requests as well as decks (a two-level bootstrap), so the interval also covers which templates were tested and the one-round medium runs get an interval.
More rounds: medium and the tuned-template reference runs ran once per request.
Intervals for time and cost: minutes per slide now carry one; minutes and cost per deck do not yet.
A fresh set of new templates: the sprint fixes were informed by the new-template decks of this run, so measuring them needs templates nobody has looked at.
pi: merge the GPT-5.1 setup for pi with a pi version of the output checks, and rerun pi on master.
A second grader: a human side-by-side review of the 31 deck pairs the grader was least sure about is in progress.
Method notes
Benchmark: pptx setup ladder, runs of 23 to 26 October 2026 on master e73d72aef1. 265 deck-building runs; 17 cut by stream errors were rerun, leaving 248 decks graded blind (175 on new templates, 73 on tuned templates).
Rounds: GPT-5.1 high (Codex and pi) 3 rounds on new templates and 2 on tuned ones; Claude Opus 5.5 and GPT-6.1 Sol 2 and 1; GPT-5.1 medium 1 round of every request per setup.
Grader: one blind grader agent per deck, pptx-deck-review skill at 22476c18f9. Rulings: a deck that does not open counts as 0 client-ready slides; two high decks that removed a requested example.com address to clear a false alarm pass by ruling.
Weighting: mean over a request’s decks, then over requests, so setups with fewer rounds compare fairly and each request weighs the same.
Intervals: 95% bootstrap that resamples decks within each request. A setup with one deck per request (all medium runs, and the reference models on tuned templates) gets a zero-width interval from this bootstrap, so none is shown. Results over all 13 requests are not reported, because the new and tuned sets are kept apart and the benchmark computes no interval for a pooled value.
Time and cost: time is wall-clock per deck, one GPT-5.1 deck at a time to stay inside the Azure quota. Cost is priced request by request at list prices (GPT-5.1 USD 1.25 input, 0.125 cached, 10 output per million tokens); GPT-5.1 is billed mostly at the cached rate, so a 20-minute deck still costs about USD 1.25.
Minutes per slide: minutes per requested slide is each deck's wall-clock minutes divided by the slides its request asked for (5.4 on average on new templates, 6.4 on tuned templates), mean over a request's decks and then over requests. Minutes per client-ready slide is a ratio of two request-weighted means, minutes per deck over client-ready slides per deck, because some decks have no client-ready slide and a per-deck ratio is undefined. Both intervals come from the same bootstrap as the accuracy measures. Each turn also has a fixed part (template selection and reading), so short decks cost more per slide.
Harnesses: GPT-5.1 and GPT-6.1 Sol on Codex (Azure), Claude Opus 5.5 on Claude Code (Vertex), GPT-5.1 on pi. The pi tools-and-rules setup is a local configuration, not on master.