How to read the benchmark results | ERPBenchmarks

Questions: 21

The shared question set

Models: 21

Shown in the batch graph

A score is shown only when a model has an assessed answer for every question. Missing, failed, or held answers are not converted to zero; check each expanded model row for its coverage.

Start here

One score scale, separate measures

Quality is rubric points earned from the first completed answer for each question. The 0–100 score appears only after every question is assessed; there is no partial-coverage dot, averaging across source runs, or best-answer selection.

Completion describes whether an answer completed, not whether it was correct. Failed attempts remain in the attempt and failure accounting but receive no invented grade.

Latency is the median elapsed time for selected completed answers. It excludes failed attempts and is not an end-to-end task-time or speed ranking.

Cost retains reported attempt costs. A known-cost subtotal is partial when any attempt has unknown billing; unknown is not zero, and failures may still incur charges.

The model graph uses native <details> disclosure controls that work with keyboard activation and without JavaScript.

For readers who want the details

Study design and limits

Questions, scope, and candidate access

The fixed Adaptive Planning question set covers formula, configuration or scenario, and integration-workflow tasks. It is a limited sample, not a representative measure of every Workday process, product release, or tenant. The broader category outline is not evidence that every listed topic was evaluated.

Candidates received no reference answers, browsing, documentation retrieval, plugins, tools, credentials, or tenant records. No submitted formula or generated code was executed. Public Workday documentation informed rubric review, but the applicable product release is not established by that fact.

This publication has no results for Finance, Reporting and Integrations. A topic page is not evidence of an evaluated result.

Why request conditions are mixed

Batch 1 combines technical source records collected under different request limits, timing policies, and question-selection rules. Original, recovery, supplement, and any additional completion records are not matched equal-compute trials; they support one question-level score view rather than separate rankings for each run. Each technical record below retains its own conditions and limitations.

The per-model score uses the first completed answer for each model/question pair, even when that answer came from a later source run. Later successful answers do not replace an already completed answer to seek a higher score. Failed attempts and all other actual attempts remain represented in attempt, status, and cost accounting.

Original technical evidence 1

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-original-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Recovery technical evidence 1

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-recovery-01-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Recovery technical evidence 2

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-recovery-02-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Recovery technical evidence 3

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-recovery-03-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Recovery technical evidence 4

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-recovery-04-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Recovery technical evidence 5

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-recovery-05-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Recovery technical evidence 6

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-recovery-06-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Recovery technical evidence 7

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-recovery-07-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Recovery technical evidence 8

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-recovery-08-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Recovery technical evidence 9

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-recovery-09-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Recovery technical evidence 10

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-recovery-10-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Recovery technical evidence 11

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-recovery-11-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Recovery technical evidence 12

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-recovery-12-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Recovery technical evidence 13

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-recovery-13-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Supplement technical evidence 1

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-supplement-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Completion technical evidence 1

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-completion-deepseek-v41-flash-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Completion technical evidence 2

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-completion-hy4-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Completion technical evidence 3

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-completion-mimo-v25-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

Completion technical evidence 4

Technical evidence retained for coverage and audit; batch quality uses first-completed answers.

Source run: adaptive-planning-completion-mimo-novita-2026-09
Published: 2026-09-28
Product version: Unknown release; no tenant validation
Track: Model only
Questions: 21
Environment and request conditions: The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Review method: Provisional blinded AI-assisted rubric review. Documentation-backed credit includes calibrated partial credit without treating undocumented alternatives as proven false. Focused cross-answer calibration covered archived versions, prior-period recurrence and import modes. No independent expert or tenant validation was performed.
Coverage note: This technical source is combined into the batch; its model rows are not separate comparisons.

Limitations

Scores are provisional documentation-backed rubric results, not independently validated ERP accuracy or statistically established rankings.
The first completed answer is selected per model and question, never the highest-scoring retry. Original, recovery and completion requests used different limits; this is not an equal-compute comparison.
Latency is the median of selected completed-answer elapsed times. Costs cover all candidate API attempts, including failures; unknown charges keep total cost unknown. Assistant review and infrastructure costs are not measured.
Unverified behavior receives no unsupported credit under the owner-approved rule; this does not prove an undocumented alternative is false.
Original collection requested 8192 output tokens and a 180-second timeout. The first 61 attempts used socket-inactivity timeout enforcement; the remaining 359 used a wall-clock deadline. This is not a uniform-deadline cohort.
Failure-selected recovery only: originally completed answers were not repeated. Expanded 32768-token and 600-second request limits, with paced requests; historical owner stops and noncompletion cutoffs remain in the attempt evidence.
Supplemental model collection used 32768-token and 600-second limits. A narrow owner-approved retention exception applied only to these nonconfidential benchmark prompts; provider retention was possible, training was disallowed, and no tools or credentials were supplied.
Missing-answer completion only; earlier completed answers are retained. Requests used 32768-token and 600-second limits, fixed original provider routes, ZDR and no-training restrictions. Three consecutive noncompletions paused dispatch for diagnosis; no model was silently excluded.
The final 17 missing MiMo answers used owner-approved Novita after DeepInfra rate limits, with the same model and frozen questions. Requests retained ZDR, no training, no fallback, 32768-token and 600-second limits. FP8 was catalog metadata, not independently verified inference precision.
Quality scores use the first completed answer per model/question; incomplete assessment coverage is unscored.

How the score is calculated and reviewed

For the 21-question set, the displayed score is 100 × earned rubric points ÷ 210 possible points, using each question's first completed answer. A score is available only when every question has an assessed grade. Completed but held answers, unanswered questions, and failures are not assigned zero grades; a model without full assessed coverage remains unscored.

Quality review is provisional and AI-assisted; the method and limitations are listed in the batch and source records. Provisional AI-assisted review is not independent expert validation or tenant testing. No inter-rater reliability statistic is implied unless explicitly reported.

Equivalent supported approaches can receive credit under the published rubric. A numerical score is rubric performance on this question set, not the percentage of fully correct answers, the chance a formula will run, or purchasing advice.

Coverage, failures, latency, and cost

Model rows distinguish completed answers, assessed answers, held answers, and missing questions. Full score coverage requires assessment of the entire shared set; a held or missing item prevents a full-set score. This page does not infer that every model is fully assessed from the question-set size.

Attempt and status totals preserve actual requests, including failures and requests that did not supply the selected answer. Median latency uses selected completed requests only, so excluding failures can make it look faster. It is not a combined quality/efficiency measure.

Known cost is the sum of priced attempts; if any attempt cost is unknown, the total remains unknown. An explicit reported zero is distinct from missing billing data. These observations are not a settled invoice and do not combine quality, completion, latency, or cost into an overall winner or efficiency rank.

Download the public record

Download the public JSON record. It retains allowlisted question-level batch summaries and technical source records for traceability, not raw answers, private references, credentials, or identity maps. The technical records support this single batch and are not separate benchmark pages.

Available result pages

Adaptive Planning · Batch 121 questions · 21 models