A separate benchmark of 525 physician-authored clinical conversations scored with a length-adjusted metric. Independent and vendor cohorts are shown separately.
Latest available model results
22 models with scores
Checked Oct 24, 2026. One entry per model version from reviewed 2026 reports or undated current sources; historical cohorts are excluded. Completed HealthBench AI retests replace the corresponding publisher scores; pending runs do not replace anything. Original scores remain in source history. Other sources are selected by publication recency. Entries are shown highest reported score first within each metric type. No model is pinned or promoted.
Latest available evidence does not mean the latest model release. Publication dates, inference settings and metrics remain visible. Scores from different runs are not directly comparable. No automatic live refresh.
Length-adjusted scores · high to low · different evaluation protocols remain non-comparable
Nox Orion 2.1
Nox
69.52895% CI 65.81–73.03Length-adjusted score (0–100)
Evaluated by HealthBenchAIOct 23, 2026 run; arithmetic checked Oct 24View source ↗
Evaluation details & limitations
Nox Orion 2.1 — HealthBenchAI team evaluation (Oct 23, 2026)
Arithmetic checked only; 525 tasks include 40 development tasks. Raw 72.833; declared holdout adjusted 69.844. Tool-assisted product configuration, not bare-model testing.
525 tasks; single tool-assisted product-pipeline run with planning, live web evidence, drafting, audit and revision. Reported grader GPT-5.4 low. Adjustment: raw − 1.47 × (characters − 2000) / 500. Three failed tasks recovered.
The HealthBenchAI team reports running this evaluation. The retained run package was checked by recomputing its score arithmetic; this later audit did not repeat model inference or independently regrade answers. This is a team-operated evaluation, not third-party certification.
The package provides per-task scores and character counts, not Professional answers or criterion-level grader records.
40 tasks were used in development; 485 declared held out. Holdout adjusted mean 69.844, 95% task-bootstrap interval 66.048–73.519.
No preferential placement. Tool use, single-run sampling and development exposure differ from comparison reports; numeric score order is not proof of cross-report superiority.
95% task-bootstrap interval, 10,000 resamples. Not run-to-run or grader uncertainty; not a clinical validation.
Official · official vendor reportJun 9, 2026View source ↗
Evaluation details & limitations
Anthropic system-card comparison — includes restricted-access Mythos models
Restricted access · Anthropic-reported, five trials · not a HealthBench AI retest.
Claude Opus 4.8 judge; adaptive thinking at max effort; average of 5 trials; no tools or customized system prompts.
Mythos 5 and Mythos Preview have explicitly labelled scores in the official source figure. Access is restricted; inclusion does not imply general API or consumer availability.
Anthropic identifies Mythos 5.1 as the newer version. The reviewed 5.1 card’s §8.17.2 reports Fable 5.1, not Mythos 5.1: its 62.1 length-adjusted score must not be reassigned to Mythos. No Mythos 5.1 Professional score is entered without a source explicitly naming that model.
These are Anthropic-reported results, not HealthBench AI retests. They do not replace our completed Fable 5.1 evaluation.
The cited GPT-5.5 comparator uses OpenAI’s own grader and is omitted from this Anthropic-run group.
The figure shows 95% confidence intervals, but exact numeric bounds are not provided here; none have been estimated or invented. Different judges/runs remain separate.
Official · official vendor reportJun 9, 2026View source ↗
Evaluation details & limitations
Anthropic system-card comparison — includes restricted-access Mythos models
Earlier preview version · Anthropic-reported, five trials · distinct from Mythos 5 and 5.1; general availability is not implied.
Claude Opus 4.8 judge; adaptive thinking at max effort; average of 5 trials; no tools or customized system prompts.
Mythos 5 and Mythos Preview have explicitly labelled scores in the official source figure. Access is restricted; inclusion does not imply general API or consumer availability.
Anthropic identifies Mythos 5.1 as the newer version. The reviewed 5.1 card’s §8.17.2 reports Fable 5.1, not Mythos 5.1: its 62.1 length-adjusted score must not be reassigned to Mythos. No Mythos 5.1 Professional score is entered without a source explicitly naming that model.
These are Anthropic-reported results, not HealthBench AI retests. They do not replace our completed Fable 5.1 evaluation.
The cited GPT-5.5 comparator uses OpenAI’s own grader and is omitted from this Anthropic-run group.
The figure shows 95% confidence intervals, but exact numeric bounds are not provided here; none have been estimated or invented. Different judges/runs remain separate.
HealthBench AI matched-protocol run (Oct 25, 2026)
Raw 67.6; gpt-6.1-sol, effort max; 525/525 assessed (525 rubric-graded including 8 token-limited completions, 2 of them empty finals; 0 API refusal no-answer fixed zeros under the disclosed local policy).
525 tasks per model, one run each, GPT-5.4 (low) grader with one request per rubric item; raw − 1.47 × (characters − 2000) / 500. Both models evaluated under the same harness. Exception: an explicit API refusal with empty content and zero output tokens is assessed as a disclosed fixed zero (raw and adjusted), with no length bonus and no invented rubric verdicts; visible refusal text is graded normally.
Self-operated by HealthBench AI (REAGEN, Inc.). Not a third-party certification, not an OpenAI-certified evaluation, and separate from Artificial Analysis and vendor reports.
One run per task, not four repeats; Fable high and GPT max are different effort configurations.
Tested by HealthBench AI under one shared API harness with no web search or external tools. This does not claim equivalence to the publisher’s internal environment.
Each completed retest replaces that model version’s publisher score in current coverage. Pending models retain their previous source-backed scores until their full run completes. Original reports remain in source history.
Raw prompts and responses stay private; hashes in the evidence file allow integrity checks only.
API no-answer refusals are scored zero under a disclosed local failure policy and listed as api_refusal_no_answer; this is not claimed as an official automatic-zero rule. Safeguards were not bypassed.
Token-limited completions are rubric-graded exactly as generated at the common 16,384-output-token cap. Empty final messages are explicitly marked rubric_graded_empty_truncated, receive actual verdicts and normal length adjustment, and are not regenerated or confused with refusals.
95% task-bootstrap interval; not run-to-run or grader uncertainty.
HealthBench AI matched-protocol run (Oct 25, 2026)
Raw 64.6; claude-fable-5-1, effort high; 525/525 assessed (519 rubric-graded including 2 token-limited completions, 0 of them empty finals; 6 API refusal no-answer fixed zeros under the disclosed local policy).
525 tasks per model, one run each, GPT-5.4 (low) grader with one request per rubric item; raw − 1.47 × (characters − 2000) / 500. Both models evaluated under the same harness. Exception: an explicit API refusal with empty content and zero output tokens is assessed as a disclosed fixed zero (raw and adjusted), with no length bonus and no invented rubric verdicts; visible refusal text is graded normally.
Self-operated by HealthBench AI (REAGEN, Inc.). Not a third-party certification, not an OpenAI-certified evaluation, and separate from Artificial Analysis and vendor reports.
One run per task, not four repeats; Fable high and GPT max are different effort configurations.
Tested by HealthBench AI under one shared API harness with no web search or external tools. This does not claim equivalence to the publisher’s internal environment.
Each completed retest replaces that model version’s publisher score in current coverage. Pending models retain their previous source-backed scores until their full run completes. Original reports remain in source history.
Raw prompts and responses stay private; hashes in the evidence file allow integrity checks only.
API no-answer refusals are scored zero under a disclosed local failure policy and listed as api_refusal_no_answer; this is not claimed as an official automatic-zero rule. Safeguards were not bypassed.
Token-limited completions are rubric-graded exactly as generated at the common 16,384-output-token cap. Empty final messages are explicitly marked rubric_graded_empty_truncated, receive actual verdicts and normal length adjustment, and are not regenerated or confused with refusals.
95% task-bootstrap interval; not run-to-run or grader uncertainty.
Official · official vendor reportJun 9, 2026View source ↗
Evaluation details & limitations
Anthropic system-card comparison — includes restricted-access Mythos models
Claude Opus 4.8 judge; adaptive thinking at max effort; average of 5 trials; no tools or customized system prompts.
Mythos 5 and Mythos Preview have explicitly labelled scores in the official source figure. Access is restricted; inclusion does not imply general API or consumer availability.
Anthropic identifies Mythos 5.1 as the newer version. The reviewed 5.1 card’s §8.17.2 reports Fable 5.1, not Mythos 5.1: its 62.1 length-adjusted score must not be reassigned to Mythos. No Mythos 5.1 Professional score is entered without a source explicitly naming that model.
These are Anthropic-reported results, not HealthBench AI retests. They do not replace our completed Fable 5.1 evaluation.
The cited GPT-5.5 comparator uses OpenAI’s own grader and is omitted from this Anthropic-run group.
The figure shows 95% confidence intervals, but exact numeric bounds are not provided here; none have been estimated or invented. Different judges/runs remain separate.
No public HealthBench Professional score verified in our reviewed sources for these exact versions. This is a research gap, not a zero or proof that no result exists. Older model scores are never substituted.
Kimi K3.1 Pro · Not verified Muse Wonder Full · Not verified
Metric: Length-adjusted score (0–100). Protocol: 525 tasks per model, one run each, GPT-5.4 (low) grader with one request per rubric item; raw − 1.47 × (characters − 2000) / 500. Both models evaluated under the same harness. Exception: an explicit API refusal with empty content and zero output tokens is assessed as a disclosed fixed zero (raw and adjusted), with no length bonus and no invented rubric verdicts; visible refusal text is graded normally.
Self-operated by HealthBench AI (REAGEN, Inc.). Not a third-party certification, not an OpenAI-certified evaluation, and separate from Artificial Analysis and vendor reports.
One run per task, not four repeats; Fable high and GPT max are different effort configurations.
Tested by HealthBench AI under one shared API harness with no web search or external tools. This does not claim equivalence to the publisher’s internal environment.
Each completed retest replaces that model version’s publisher score in current coverage. Pending models retain their previous source-backed scores until their full run completes. Original reports remain in source history.
Raw prompts and responses stay private; hashes in the evidence file allow integrity checks only.
API no-answer refusals are scored zero under a disclosed local failure policy and listed as api_refusal_no_answer; this is not claimed as an official automatic-zero rule. Safeguards were not bypassed.
Token-limited completions are rubric-graded exactly as generated at the common 16,384-output-token cap. Empty final messages are explicitly marked rubric_graded_empty_truncated, receive actual verdicts and normal length adjustment, and are not regenerated or confused with refusals.
95% task-bootstrap interval; not run-to-run or grader uncertainty.
Direct answer
In HealthBench AI's self-operated run (Oct 25, 2026), GPT-6.1 Sol (max) scored 64.6 length-adjusted on 525/525 tasks; Claude Fable 5.1 (high) scored 58.0 length-adjusted on 525/525 tasks. Both models used one harness. Not third-party certification.
Source-backed snapshots, last checked Oct 24, 2026; not an exhaustive census. Counts include reported configurations and repeat appearances across sources, not unique base models. A missing model is not a zero. Cohorts are never combined; vendor-reported does not mean independently validated.
Results in selected cohort
2 rows
2 results
HealthBench Professional: HealthBench AI matched-protocol run (Oct 25, 2026)
Rank
Model
Organization
Score
Metric
Source date
Notes
Source
01
GPT-6.1 Sol (max)
OpenAI
64.634
95% CI 60.78–68.35
Length-adjusted score (0–100)
Oct 25, 2026
Raw 67.6; gpt-6.1-sol, effort max; 525/525 assessed (525 rubric-graded including 8 token-limited completions, 2 of them empty finals; 0 API refusal no-answer fixed zeros under the disclosed local policy).
Raw 64.6; claude-fable-5-1, effort high; 525/525 assessed (519 rubric-graded including 2 token-limited completions, 0 of them empty finals; 6 API refusal no-answer fixed zeros under the disclosed local policy).
7 cohorts, 35 result rows, 33 distinct model names. Many models appear in more than one cohort; rows are not unique models. Source-backed snapshots, last checked Oct 24, 2026; not an exhaustive census. Counts include reported configurations and repeat appearances across sources, not unique base models. A missing model is not a zero. Cohorts are never combined; vendor-reported does not mean independently validated.
HealthBench AI matched-protocol run (Oct 25, 2026) · Length-adjusted score (0–100) · default
Metric: Length-adjusted score (0–100). Protocol: 525 tasks per model, one run each, GPT-5.4 (low) grader with one request per rubric item; raw − 1.47 × (characters − 2000) / 500. Both models evaluated under the same harness. Exception: an explicit API refusal with empty content and zero output tokens is assessed as a disclosed fixed zero (raw and adjusted), with no length bonus and no invented rubric verdicts; visible refusal text is graded normally.
Self-operated by HealthBench AI (REAGEN, Inc.). Not a third-party certification, not an OpenAI-certified evaluation, and separate from Artificial Analysis and vendor reports.
One run per task, not four repeats; Fable high and GPT max are different effort configurations.
Tested by HealthBench AI under one shared API harness with no web search or external tools. This does not claim equivalence to the publisher’s internal environment.
Each completed retest replaces that model version’s publisher score in current coverage. Pending models retain their previous source-backed scores until their full run completes. Original reports remain in source history.
Raw prompts and responses stay private; hashes in the evidence file allow integrity checks only.
API no-answer refusals are scored zero under a disclosed local failure policy and listed as api_refusal_no_answer; this is not claimed as an official automatic-zero rule. Safeguards were not bypassed.
Token-limited completions are rubric-graded exactly as generated at the common 16,384-output-token cap. Empty final messages are explicitly marked rubric_graded_empty_truncated, receive actual verdicts and normal length adjustment, and are not regenerated or confused with refusals.
95% task-bootstrap interval; not run-to-run or grader uncertainty.
In HealthBench AI's self-operated run (Oct 25, 2026), GPT-6.1 Sol (max) scored 64.6 length-adjusted on 525/525 tasks; Claude Fable 5.1 (high) scored 58.0 length-adjusted on 525/525 tasks. Both models used one harness. Not third-party certification.
GPT-6.1 Sol (max) (OpenAI): 64.634 — Raw 67.6; gpt-6.1-sol, effort max; 525/525 assessed (525 rubric-graded including 8 token-limited completions, 2 of them empty finals; 0 API refusal no-answer fixed zeros under the disclosed local policy).
Claude Fable 5.1 (high) (Anthropic): 57.971 — Raw 64.6; claude-fable-5-1, effort high; 525/525 assessed (519 rubric-graded including 2 token-limited completions, 0 of them empty finals; 6 API refusal no-answer fixed zeros under the disclosed local policy).
OpenAI GPT-6.1 Sol system card, Table 7 (official) · Length-adjusted score (0–100)
Publication date is not printed on the retrieved page.
Judge and setup differ from Anthropic and Artificial Analysis cohorts; never mixed.
In the OpenAI GPT-6.1 Sol system card (Table 7, checked Oct 24, 2026), GPT-6 Astra has the highest length-adjusted HealthBench Professional score at 64.7, followed by GPT-6.1 Sol at 64.2.
GPT-6 Astra (OpenAI): 64.7
GPT-6.1 Sol (OpenAI): 64.2
GPT-6 Sol (OpenAI): 60.8
GPT-6 Luna (OpenAI): 60.8
GPT-5.6 Sol (OpenAI): 60.5
GPT-5.6 Terra (OpenAI): 57.7
GPT-5.6 Luna (OpenAI): 55.7
GPT-5.5 (OpenAI): 51.8
Anthropic Claude Sonnet 5.5 system card, §8.15 (Oct 2026) · Length-adjusted score (0–100)
Metric: Length-adjusted score (0–100). Protocol: Anthropic-run cross-model comparison. Judge: Claude Opus 4.8. All models at max effort; Claude safety classifiers enabled; OpenAI model sampled via public API with no system prompt. 1.47 points per 500 characters above or below 2,000.
Different judge from OpenAI and Artificial Analysis cohorts, so never mixed with them.
Raw (unadjusted) scores, notes only: GPT-6 Astra 74.0, Claude Sonnet 5.5 77.1, Claude Opus 5.5 77.1, Claude Fable 5.1 74.2.
In Anthropic’s Claude Sonnet 5.5 system card (§8.15, Oct 2026), GPT-6 Astra has the highest length-adjusted HealthBench Professional score at 70.3, followed by Claude Sonnet 5.5 (max) at 69.2, using a Claude Opus 4.8 judge.
GPT-6 Astra (OpenAI): 70.3 — Raw 74.0
Claude Sonnet 5.5 (max) (Anthropic): 69.2 — Raw 77.1
Claude Opus 5.5 (max) (Anthropic): 65.6 — Raw 77.1
Claude Fable 5.1 (max) (Anthropic): 62.1 — Raw 74.2
Metric: Length-adjusted score (0–100). Protocol: 525 clinician conversations, GPT-5.4 (low) judge, 4 repeats; subtract 1.47 points per 500 characters above 2,000; add the same amount for shorter responses.
All 14 scores visually checked against Artificial Analysis’s original published chart, not only a secondary transcription.
Independent evaluation by Artificial Analysis, not by HealthBench AI; not directly comparable with the OpenAI paper cohort.
In Artificial Analysis’s independent HealthBench Professional evaluation (Oct 19, 2026), GPT-6 Astra (Max) has the highest length-adjusted score at 66.0, followed by GPT-6.1 Sol (Max) at 65.1 and Claude Sonnet 5.5 (Max) at 63.0. All 14 scores are verified against the evaluator’s original chart.
GPT-6 Astra (Max) (OpenAI): 66.0
GPT-6.1 Sol (Max) (OpenAI): 65.1
Claude Sonnet 5.5 (Max, Default Fallback) (Anthropic): 63.0
GPT-6 Sol (Max) (OpenAI): 62.3
GPT-6 Luna (Max) (OpenAI): 61.9
Claude Opus 5.5 (Max, Default Fallback) (Anthropic): 59.9
Claude Haiku 5.5 (Max) (Anthropic): 58.4
Claude Fable 5.1 (Max, Default Fallback) (Anthropic): 57.8
Grok 4.7 (Xhigh) (SpaceXAI): 55.8
Gemini 4 Argon (High) (Google): 55.1
Kimi K3 (Max) (Kimi): 53.2
GLM-5.3 (Max) (Z AI): 52.8
Gemini 3.8 Flash (High) (Google): 51.7
DeepSeek V4.1 Flash (Max) (DeepSeek): 50.7
OpenAI HealthBench Professional paper (Apr 2026) · Length-adjusted score (0–100)
Metric: Length-adjusted score (0–100). Protocol: Length-adjusted overall score as reported by OpenAI; includes system configurations (ChatGPT for Clinicians, browsing), not only base models.
OpenAI-run; model configurations, sampling and evaluation implementation can differ from the Artificial Analysis cohort.
In the April 2026 OpenAI HealthBench Professional paper, GPT-5.4 in ChatGPT for Clinicians reported the highest listed length-adjusted score at 59.0; the GPT-5.4 base model scored 48.1.
GPT-5.4 in ChatGPT for Clinicians (OpenAI): 59.0 — ChatGPT for Clinicians system
GPT-5.4 (OpenAI): 48.1 — Base model
GPT-5.4 with browsing (OpenAI): 45.8 — Browsing harness
Metric: Length-adjusted score (0–100). Protocol: 525 tasks; single tool-assisted product-pipeline run with planning, live web evidence, drafting, audit and revision. Reported grader GPT-5.4 low. Adjustment: raw − 1.47 × (characters − 2000) / 500. Three failed tasks recovered.
The HealthBenchAI team reports running this evaluation. The retained run package was checked by recomputing its score arithmetic; this later audit did not repeat model inference or independently regrade answers. This is a team-operated evaluation, not third-party certification.
The package provides per-task scores and character counts, not Professional answers or criterion-level grader records.
40 tasks were used in development; 485 declared held out. Holdout adjusted mean 69.844, 95% task-bootstrap interval 66.048–73.519.
No preferential placement. Tool use, single-run sampling and development exposure differ from comparison reports; numeric score order is not proof of cross-report superiority.
95% task-bootstrap interval, 10,000 resamples. Not run-to-run or grader uncertainty; not a clinical validation.
Nox Orion 2.1 reports 69.528 length-adjusted (raw 72.833), with a recomputed 95% task-bootstrap interval of 65.814–73.033. This is a submitted self-test whose arithmetic was checked, not independent verification or an established first-place ranking.
Nox Orion 2.1 (Nox): 69.528 — Arithmetic checked only; 525 tasks include 40 development tasks. Raw 72.833; declared holdout adjusted 69.844. Tool-assisted product configuration, not bare-model testing.
Metric: Length-adjusted score (0–100). Protocol: Claude Opus 4.8 judge; adaptive thinking at max effort; average of 5 trials; no tools or customized system prompts.
Mythos 5 and Mythos Preview have explicitly labelled scores in the official source figure. Access is restricted; inclusion does not imply general API or consumer availability.
Anthropic identifies Mythos 5.1 as the newer version. The reviewed 5.1 card’s §8.17.2 reports Fable 5.1, not Mythos 5.1: its 62.1 length-adjusted score must not be reassigned to Mythos. No Mythos 5.1 Professional score is entered without a source explicitly naming that model.
These are Anthropic-reported results, not HealthBench AI retests. They do not replace our completed Fable 5.1 evaluation.
The cited GPT-5.5 comparator uses OpenAI’s own grader and is omitted from this Anthropic-run group.
The figure shows 95% confidence intervals, but exact numeric bounds are not provided here; none have been estimated or invented. Different judges/runs remain separate.
Anthropic’s June 9, 2026 system card reports Claude Mythos 5 at 66.0, Mythos Preview at 64.7 and Opus 4.8 at 56.9 length-adjusted on HealthBench Professional. Mythos access is restricted. These are vendor-reported scores, not HealthBench AI tests. The newer Mythos 5.1 is distinct; the reviewed 5.1 card’s Professional score of 62.1 is explicitly for Fable 5.1 and is not attributed to Mythos 5.1.
Claude Mythos 5 (max) (Anthropic): 66.0 — Restricted access · Anthropic-reported, five trials · not a HealthBench AI retest.
Claude Mythos Preview (max) (Anthropic): 64.7 — Earlier preview version · Anthropic-reported, five trials · distinct from Mythos 5 and 5.1; general availability is not implied.
Claude Opus 4.8 (max) (Anthropic): 56.9
Definition
HealthBench Professional is a distinct benchmark, not a subset of HealthBench. It contains 525 tasks written by physicians to reflect professional clinical work such as documentation, consultation and clinical reasoning.
Scores are length-adjusted to correct for graders favouring longer answers. Judges, repeats and penalty sizes differ between evaluators, so cohorts are never merged.
Source and date
Default cohort: HealthBenchAI Professional evaluation methodology (Oct 25, 2026; last checked Oct 24, 2026). Cohorts are never combined and nothing updates automatically. HealthBench Professional and model names belong to their respective owners; REAGEN, Inc. is not affiliated with them.
HealthBench AI
Muse Wonder Full · Not verified