Client-Ready Decks: Benchmarks

2 min read

Accuracy vs cost and new vs tuned templates

All setups from both evaluation runs. Colour is the model, shape the run; whiskers are 95% intervals where the run has one. Times are not shown: the November run ran on QA with up to eight decks at once, so its minutes are not comparable with October.

decks_clean_vs_cost_all.png

GPT-6.1 models build client-ready decks with the standard PowerPoint skill alone. On templates nobody tuned on, GPT-6.1 Terra reaches 96 to 100% client-ready slides at USD 0.36 to 0.47 per deck, level with GPT-6.1 Sol (98%) and Claude Opus 5.5 (99%, USD 1.60). GPT-5.1 needs the optimised setup: it lifts GPT-5.1 high from 26% to 72%.

decks_new_vs_tuned_all.png

On the tuned templates, where the GPT-5.1 optimised setup was developed, that setup is ahead (97%). GPT-6.1 Sol and Terra run the standard skill there and reach 80 to 85%: they leave template sample icons or skip the template's photo cover, which the optimised setup checks for.

Key facts

  • What we test: how well Unique AI Conduct builds client-ready PowerPoint decks in a bank's own corporate template, end to end, as a relationship manager (RM) would request them.

  • Requests: 13 requests in English, German and French: 8 on 4 company templates the setup was not developed on (new templates) and 5 on 4 templates it was developed on (tuned templates). All figures in the requests are fictitious.

  • Score: client-ready slides, the share of requested slides that need no fix before the deck goes to the client. Every deck is graded blind, slide by slide, against a 15-item checklist.

  • Runs: 23 to 26 October 2026, 248 decks with GPT-5.1, GPT-6.1 Sol and Claude Opus 5.5; 12 November 2026, 57 decks with GPT-6.1 Terra at three reasoning efforts, GPT-6.1 Sol as the reference and the GPT-5.1 optimised setup on the tuned templates.

  • Consistency: GPT-6.1 Sol medium reaches 98% on the new templates in both runs, so the requests and the grading give the same answer across runs.

  • Recommendation: GPT-6.1 Terra at medium or high reasoning effort where the deployment offers it: as accurate as the frontier reference models on new templates, at the lowest cost per deck. Which models may process client-identifying data is set per deployment; see the <![CDATA[LLM Availability Overview]]>.

  • Caveats: the November run has one deck per request, so a difference of one deck is within noise and no interval is given. Cost per deck of the same model differs between runs (GPT-6.1 Sol USD 0.54 and 0.77), so compare cost within a run. Two tuned templates are newer versions in the November run, so tuned-template results are not directly comparable across runs. One grader, written by the team that built the setup.

Evaluations

Last updated