Question Extraction Benchmarks
2 min read
Measured results for Question Extraction. These pages are benchmarks: what was tested, on which model, and what to expect. Extraction turns an uploaded questionnaire (PDF, DOCX or XLSX) into the list of questions the Agentic Table answers, with each question's guidance and section context.
What users get. When extraction completes on the standard route, the best models capture 93–94% of the questions (GPT-6 Astra 94%, GPT-6.1 Sol 94%, GPT-6 Sol 93%, GPT-5.4 93%), and the differences between them are not significant. GPT-4o (no reasoning, production default) captures 61%. The models differ in how often they complete a document: GPT-6 Sol has the highest recall of the models that complete all 18 standard documents, while GPT-6 Astra and GPT-6.1 Sol do not complete 6 and 7 of them. Five documents are larger than the standard route reads in one request; they are left out of the comparison because their result reflects document size, not the model, and the report's notes explain how each was handled.
Extraction models compared on the 11 documents every model completes, 435 questions (October 2026). No reasoning setting is sent, so each model ran at its provider's default, which is how the product calls extraction.
Recall on the shared documents against mean list-price cost per document. Models with complete cost evidence on the shared documents only (GPT-6.1 Sol left out). Cheapest at each level of recall: GPT-6 Luna, GPT-6 Sol and GPT-6 Astra. GPT-5.6 Luna and GPT-5 Mini are not plotted: their workbook runs did not start, so they have no result on the 8 workbooks in this set. Their PDF/DOCX results are in the report.
Recall on the shared documents against mean extraction time per document (log scale). GPT-5.6 Luna and GPT-5 Mini are not plotted: their workbook runs did not start, so they have no result on the 8 workbooks in this set. Their PDF/DOCX results are in the report.
GPT-6.1 Sol, GPT-6 Astra, GPT-6 Luna and GPT-5.4 did not complete 7, 6, 1 and 1 of the standard documents, which moves them to the left; GPT-4o completed all 18 but captures fewer of their questions. GPT-5.6 Luna and GPT-5 Mini are not plotted: their workbook runs did not start, so they have no result on the 8 workbooks in this set. Their PDF/DOCX results are in the report.