HealthBench AIOverviewSubmit a resultA 1,000-example challenging subset of HealthBench (5,000 conversations). Cohorts differ in scoring: some are length-adjusted, some raw, so each is shown separately.
Checked Oct 24, 2026. One entry per model version from reviewed 2026 reports or undated current sources; historical cohorts are excluded. Sources are selected by publication recency. Entries are shown highest reported score first within each metric type. No model is pinned or promoted.
Latest available evidence does not mean the latest model release. Publication dates, inference settings and metrics remain visible. Scores from different runs are not directly comparable. No automatic live refresh.
OpenAI GPT-6 Astra system card, Table 6 (Sep 2026)
OpenAI-run HealthBench Hard evaluation reported in Table 6. Subtract 3.92 points per 500 characters above 2,000; add the same amount for shorter responses.
Length-adjusted scores are NOT comparable with the raw HealthBench Hard scores in the other cohorts.
Vendor-reported; only OpenAI models are listed.
OpenAI GPT-6.1 Sol addendum, Table 7 (latest checked)
OpenAI-run comparison. Per 500 characters above 2,000, subtract 3.92 points; below 2,000, add the same amount.
Not comparable with unadjusted Hard scores.
OpenAI notes that comparison versions can differ from launch versions. Publication date is not printed in the retrieved page.
OpenAI GPT-6 Astra system card, Table 6 (Sep 2026)
OpenAI-run HealthBench Hard evaluation reported in Table 6. Subtract 3.92 points per 500 characters above 2,000; add the same amount for shorter responses.
Length-adjusted scores are NOT comparable with the raw HealthBench Hard scores in the other cohorts.
Vendor-reported; only OpenAI models are listed.
OpenAI GPT-6 Astra system card, Table 6 (Sep 2026)
OpenAI-run HealthBench Hard evaluation reported in Table 6. Subtract 3.92 points per 500 characters above 2,000; add the same amount for shorter responses.
Length-adjusted scores are NOT comparable with the raw HealthBench Hard scores in the other cohorts.
Vendor-reported; only OpenAI models are listed.
OpenAI GPT-6 Astra system card, Table 6 (Sep 2026)
OpenAI-run HealthBench Hard evaluation reported in Table 6. Subtract 3.92 points per 500 characters above 2,000; add the same amount for shorter responses.
Length-adjusted scores are NOT comparable with the raw HealthBench Hard scores in the other cohorts.
Vendor-reported; only OpenAI models are listed.
OpenAI GPT-6 Astra system card, Table 6 (Sep 2026)
OpenAI-run HealthBench Hard evaluation reported in Table 6. Subtract 3.92 points per 500 characters above 2,000; add the same amount for shorter responses.
Length-adjusted scores are NOT comparable with the raw HealthBench Hard scores in the other cohorts.
Vendor-reported; only OpenAI models are listed.
OpenAI GPT-6.1 Sol addendum, Table 7 (latest checked)
OpenAI-run comparison. Per 500 characters above 2,000, subtract 3.92 points; below 2,000, add the same amount.
Not comparable with unadjusted Hard scores.
OpenAI notes that comparison versions can differ from launch versions. Publication date is not printed in the retrieved page.
Nox Orion 1.4 — post-hoc length-adjusted self-test (Sep 2026)
Self-test 2026-09-05; raw 46.381. Arithmetic checked only. 3988.38 mean full-response characters.
1,000 tasks per configuration, GPT-4.1 grades supplied by submitter. All task IDs and rubric weights matched to the official Hard release. Full reply text, including safety banners, used for character counts.
Submitted self-evaluation; arithmetic and rubric joins checked, NOT independently evaluated. Grader verdicts were not independently reassessed.
Post-hoc formula: raw percentage points − 3.92 × (Python len(response_text) − 2000) / 500. This does not make the run an official OpenAI evaluation.
Ultra: 199 recorded length fields differ from full text. Recomputed full-text mean 3,988.38 characters, not recorded mean 3,868.706.
95% task-bootstrap interval, 10,000 resamples. Not run-to-run or grader uncertainty; not a clinical validation.
OpenAI GPT-6.1 Sol addendum, Table 7 (latest checked)
OpenAI-run comparison. Per 500 characters above 2,000, subtract 3.92 points; below 2,000, add the same amount.
Not comparable with unadjusted Hard scores.
OpenAI notes that comparison versions can differ from launch versions. Publication date is not printed in the retrieved page.
Baichuan M3 original report, §4.2.1 (Feb 2026)
Baichuan-run HealthBench Hard evaluation. Eight scores transcribed from Figure 7, middle panel; top three also stated in section 4.2.1. Not HealthBench Total or HealthBench-Hallu.
Model-author preprint; not independent validation.
Judge, sampling and length-adjustment parity with other vendors are not established by the cited results passage. Keep this comparison separate.
44.4 exceeds Muse Spark’s 42.8 numerically, but comes from a different evaluator. No cross-report global champion is inferred.
Meta Muse Spark evaluation methodology (raw, cross-vendor)
1,000 HealthBench Hard prompts graded by a GPT-4.1 judge, highest available reasoning setting. Per Meta, competitor figures use the most favourable of self-reported or Meta-run internal API results.
Publication date is not printed on the source.
Claude and Gemini entries are the exact versions Meta listed (Claude Opus 4.6 Max, Gemini 3.1 Pro High), not the newest releases.
Vendor comparison prepared by Meta.
Baichuan M3 original report, §4.2.1 (Feb 2026)
Baichuan-run HealthBench Hard evaluation. Eight scores transcribed from Figure 7, middle panel; top three also stated in section 4.2.1. Not HealthBench Total or HealthBench-Hallu.
Model-author preprint; not independent validation.
Judge, sampling and length-adjustment parity with other vendors are not established by the cited results passage. Keep this comparison separate.
44.4 exceeds Muse Spark’s 42.8 numerically, but comes from a different evaluator. No cross-report global champion is inferred.
Meta Muse Spark evaluation methodology (raw, cross-vendor)
1,000 HealthBench Hard prompts graded by a GPT-4.1 judge, highest available reasoning setting. Per Meta, competitor figures use the most favourable of self-reported or Meta-run internal API results.
Publication date is not printed on the source.
Claude and Gemini entries are the exact versions Meta listed (Claude Opus 4.6 Max, Gemini 3.1 Pro High), not the newest releases.
Vendor comparison prepared by Meta.
Baichuan M3 original report, §4.2.1 (Feb 2026)
Baichuan-run HealthBench Hard evaluation. Eight scores transcribed from Figure 7, middle panel; top three also stated in section 4.2.1. Not HealthBench Total or HealthBench-Hallu.
Model-author preprint; not independent validation.
Judge, sampling and length-adjustment parity with other vendors are not established by the cited results passage. Keep this comparison separate.
44.4 exceeds Muse Spark’s 42.8 numerically, but comes from a different evaluator. No cross-report global champion is inferred.
Baichuan M3 original report, §4.2.1 (Feb 2026)
Baichuan-run HealthBench Hard evaluation. Eight scores transcribed from Figure 7, middle panel; top three also stated in section 4.2.1. Not HealthBench Total or HealthBench-Hallu.
Model-author preprint; not independent validation.
Judge, sampling and length-adjustment parity with other vendors are not established by the cited results passage. Keep this comparison separate.
44.4 exceeds Muse Spark’s 42.8 numerically, but comes from a different evaluator. No cross-report global champion is inferred.
Baichuan M3 original report, §4.2.1 (Feb 2026)
Baichuan-run HealthBench Hard evaluation. Eight scores transcribed from Figure 7, middle panel; top three also stated in section 4.2.1. Not HealthBench Total or HealthBench-Hallu.
Model-author preprint; not independent validation.
Judge, sampling and length-adjustment parity with other vendors are not established by the cited results passage. Keep this comparison separate.
44.4 exceeds Muse Spark’s 42.8 numerically, but comes from a different evaluator. No cross-report global champion is inferred.
Baichuan M3 original report, §4.2.1 (Feb 2026)
Baichuan-run HealthBench Hard evaluation. Eight scores transcribed from Figure 7, middle panel; top three also stated in section 4.2.1. Not HealthBench Total or HealthBench-Hallu.
Model-author preprint; not independent validation.
Judge, sampling and length-adjustment parity with other vendors are not established by the cited results passage. Keep this comparison separate.
44.4 exceeds Muse Spark’s 42.8 numerically, but comes from a different evaluator. No cross-report global champion is inferred.
Meta Muse Spark evaluation methodology (raw, cross-vendor)
1,000 HealthBench Hard prompts graded by a GPT-4.1 judge, highest available reasoning setting. Per Meta, competitor figures use the most favourable of self-reported or Meta-run internal API results.
Publication date is not printed on the source.
Claude and Gemini entries are the exact versions Meta listed (Claude Opus 4.6 Max, Gemini 3.1 Pro High), not the newest releases.
Vendor comparison prepared by Meta.
Meta Muse Spark evaluation methodology (raw, cross-vendor)
1,000 HealthBench Hard prompts graded by a GPT-4.1 judge, highest available reasoning setting. Per Meta, competitor figures use the most favourable of self-reported or Meta-run internal API results.
Publication date is not printed on the source.
Claude and Gemini entries are the exact versions Meta listed (Claude Opus 4.6 Max, Gemini 3.1 Pro High), not the newest releases.
Vendor comparison prepared by Meta.
Baichuan M3 original report, §4.2.1 (Feb 2026)
Baichuan-run HealthBench Hard evaluation. Eight scores transcribed from Figure 7, middle panel; top three also stated in section 4.2.1. Not HealthBench Total or HealthBench-Hallu.
Model-author preprint; not independent validation.
Judge, sampling and length-adjustment parity with other vendors are not established by the cited results passage. Keep this comparison separate.
44.4 exceeds Muse Spark’s 42.8 numerically, but comes from a different evaluator. No cross-report global champion is inferred.
Meta Muse Spark evaluation methodology (raw, cross-vendor)
1,000 HealthBench Hard prompts graded by a GPT-4.1 judge, highest available reasoning setting. Per Meta, competitor figures use the most favourable of self-reported or Meta-run internal API results.
Publication date is not printed on the source.
Claude and Gemini entries are the exact versions Meta listed (Claude Opus 4.6 Max, Gemini 3.1 Pro High), not the newest releases.
Vendor comparison prepared by Meta.
Baichuan M3 original report, §4.2.1 (Feb 2026)
Baichuan-run HealthBench Hard evaluation. Eight scores transcribed from Figure 7, middle panel; top three also stated in section 4.2.1. Not HealthBench Total or HealthBench-Hallu.
Model-author preprint; not independent validation.
Judge, sampling and length-adjustment parity with other vendors are not established by the cited results passage. Keep this comparison separate.
44.4 exceeds Muse Spark’s 42.8 numerically, but comes from a different evaluator. No cross-report global champion is inferred.
No public HealthBench Hard score verified in our reviewed sources for these exact versions. This is a research gap, not a zero or proof that no result exists. Older model scores are never substituted.
Claude Sonnet 5.5 · Not verified Claude Opus 5.5 · Not verified Claude Haiku 5.5 · Not verified Claude Fable 5.1 · Not verified Gemini 4 Argon · Not verified Gemini 3.8 Flash · Not verified Grok 4.7 · Not verified
GLM-5.3 · Not verified DeepSeek V4.1 Flash · Not verified
Kimi K3.1 Pro · Not verified Muse Wonder Full · Not verifiedModel-release references: Artificial Analysis (Oct 19) ↗Artificial Analysis (Oct 23) ↗
Official · official vendor report5 rows
Source: Meta Muse Spark eval methodology · Source date: Undated; checked Oct 24, 2026 · Verified:
Metric: Raw HealthBench Hard score (0–100). Protocol: 1,000 HealthBench Hard prompts graded by a GPT-4.1 judge, highest available reasoning setting. Per Meta, competitor figures use the most favourable of self-reported or Meta-run internal API results.
In Meta’s Muse Spark evaluation methodology (undated; checked Oct 24, 2026), Muse Spark Thinking has the highest raw HealthBench Hard score at 42.8, followed by GPT-5.4 Xhigh at 40.1; Gemini 3.1 Pro High scored 20.6 and Claude Opus 4.6 Max 14.8.
Source-backed snapshots, last checked Oct 24, 2026; not an exhaustive census. Counts include reported configurations and repeat appearances across sources, not unique base models. A missing model is not a zero. Cohorts are never combined; vendor-reported does not mean independently validated.
| Rank | Model | Organization | Score | Metric | Source date | Notes | Source |
|---|---|---|---|---|---|---|---|
| 01 | Muse Spark Thinking | Meta | 42.8 | Raw HealthBench Hard score (0–100) | Undated; checked Oct 24, 2026 | — | Source |
| 02 | GPT-5.4 Xhigh | OpenAI | 40.1 | Raw HealthBench Hard score (0–100) | Undated; checked Oct 24, 2026 | — | Source |
| 03 | Gemini 3.1 Pro High | 20.6 | Raw HealthBench Hard score (0–100) | Undated; checked Oct 24, 2026 | — | Source | |
| 04 | Grok 4.2 Reasoning | xAI | 20.3 | Raw HealthBench Hard score (0–100) | Undated; checked Oct 24, 2026 | — | Source |
| 05 | Claude Opus 4.6 Max | Anthropic | 14.8 | Raw HealthBench Hard score (0–100) | Undated; checked Oct 24, 2026 | — | Source |
7 cohorts, 35 result rows, 29 distinct model names. Many models appear in more than one cohort; rows are not unique models. Source-backed snapshots, last checked Oct 24, 2026; not an exhaustive census. Counts include reported configurations and repeat appearances across sources, not unique base models. A missing model is not a zero. Cohorts are never combined; vendor-reported does not mean independently validated.
Official · official vendor report5 rows
Source: Meta Muse Spark eval methodology · Source date: Undated; checked Oct 24, 2026 · Verified:
Metric: Raw HealthBench Hard score (0–100). Protocol: 1,000 HealthBench Hard prompts graded by a GPT-4.1 judge, highest available reasoning setting. Per Meta, competitor figures use the most favourable of self-reported or Meta-run internal API results.
In Meta’s Muse Spark evaluation methodology (undated; checked Oct 24, 2026), Muse Spark Thinking has the highest raw HealthBench Hard score at 42.8, followed by GPT-5.4 Xhigh at 40.1; Gemini 3.1 Pro High scored 20.6 and Claude Opus 4.6 Max 14.8.
Muse Spark Thinking (Meta): 42.8 GPT-5.4 Xhigh (OpenAI): 40.1 Gemini 3.1 Pro High (Google): 20.6 Grok 4.2 Reasoning (xAI): 20.3 Claude Opus 4.6 Max (Anthropic): 14.8Official · official vendor report8 rows
Source: OpenAI GPT-6.1 Sol system card, Table 7 · Source date: Undated page; checked Oct 24, 2026 · Verified:
Metric: Length-adjusted rubric score (0–100). Protocol: OpenAI-run comparison. Per 500 characters above 2,000, subtract 3.92 points; below 2,000, add the same amount.
In the latest verified OpenAI GPT-6.1 Sol addendum, Table 7, GPT-6 Astra scores 36.6 and GPT-6.1 Sol scores 36.2 on length-adjusted HealthBench Hard. This is a vendor-run comparison, not a global ranking.
GPT-6 Astra (OpenAI): 36.6 GPT-6.1 Sol (OpenAI): 36.2 GPT-5.6 Sol (OpenAI): 33.1 GPT-5.6 Terra (OpenAI): 32.7 GPT-5.6 Luna (OpenAI): 32.0 GPT-5.5 (OpenAI): 31.5 GPT-6 Luna (OpenAI): 31.4 GPT-6 Sol (OpenAI): 30.1Official · official vendor report5 rows
Source: OpenAI GPT-6 Astra system card · Source date: Sep 2026 (card amendment Sep 9, 2026) · Verified:
Metric: Length-adjusted rubric score (0–100). Protocol: OpenAI-run HealthBench Hard evaluation reported in Table 6. Subtract 3.92 points per 500 characters above 2,000; add the same amount for shorter responses.
In the OpenAI GPT-6 Astra system card (Table 6, Sep 2026), GPT-6 Astra has the highest length-adjusted HealthBench Hard score at 36.6, followed by GPT-5.6 Sol at 33.1. This cohort uses a length penalty and is not comparable with raw Hard scores.
GPT-6 Astra (OpenAI): 36.6 GPT-5.6 Sol (OpenAI): 33.1 GPT-5.6 Terra (OpenAI): 32.7 GPT-5.6 Luna (OpenAI): 32.0 GPT-5.5 (OpenAI): 31.5Official · official vendor report7 rows
Source: OpenAI GPT-OSS evaluation chart · Source date: Aug 2025 · Verified:
Metric: Raw HealthBench Hard score (0–100). Protocol: Values transcribed from OpenAI’s public GPT-OSS evaluation chart; may carry small transcription error.
In the August 2025 OpenAI GPT-OSS evaluation chart, o3 had the highest listed HealthBench Hard score at 31.6, followed by gpt-oss-120b at 30.0.
o3 (OpenAI): 31.6 gpt-oss-120b (OpenAI): 30.0 o4-mini (OpenAI): 17.5 gpt-oss-20b (OpenAI): 10.8 o1 (OpenAI): 7.9 o3-mini (OpenAI): 4.0 gpt-4o (OpenAI): 0.0Official · official vendor report8 rows
Source: Baichuan M3 technical report, section 4.2.1 / Figure 7 · Source date: Feb 2026 (arXiv v1) · Verified:
Metric: Reported HealthBench Hard score (0–100). Protocol: Baichuan-run HealthBench Hard evaluation. Eight scores transcribed from Figure 7, middle panel; top three also stated in section 4.2.1. Not HealthBench Total or HealthBench-Hallu.
Baichuan’s M3 report lists Baichuan M3 at 44.4, GPT-5.2 High at 42.0 and AntAngelMed at 39.6 on HealthBench Hard. These are Baichuan-reported results, not part of Meta’s Muse Spark comparison.
Baichuan M3 (Baichuan): 44.4 GPT-5.2 High (OpenAI): 42.0 AntAngelMed (Ant Group): 39.6 Baichuan M2 (Baichuan): 34.7 Qwen3 235B Thinking 2507 (Alibaba Qwen): 25.9 DeepSeek V3.2 Thinking (DeepSeek): 21.7 Gemini 3 Pro (Google): 15.2 MedGemma 27B text-it (Google): 8.5team-run · score calculation verified1 rows
Source: HealthBenchAI: Nox Orion test results and evidence · Source date: Sep 5, 2026 run; checked Oct 24 · Verified:
Official scoring code (response length) · Official Hard dataset (rubric weights)
Metric: Post-hoc length-adjusted score (0–100). Protocol: 1,000 tasks per configuration, GPT-4.1 grades supplied by submitter. All task IDs and rubric weights matched to the official Hard release. Full reply text, including safety banners, used for character counts.
Applying the stated length adjustment to submitted Nox Orion 1.4 Ultra outputs gives 30.792 (95% task-bootstrap interval 28.965–32.577). This is a post-hoc calculation, not an independently evaluated score.
Nox Orion 1.4 Ultra (Nox): 30.792 — Self-test 2026-09-05; raw 46.381. Arithmetic checked only. 3988.38 mean full-response characters.team-run · score calculation verified1 rows
Source: HealthBenchAI: Nox Orion test results and evidence · Source date: Sep 5, 2026 run; checked Oct 24 · Verified:
Official scoring code (response length) · Official Hard dataset (rubric weights)
Metric: Raw HealthBench Hard score (0–100). Protocol: 1,000 tasks per configuration, GPT-4.1 grades supplied by submitter. All task IDs and rubric weights matched to the official Hard release. Full reply text, including safety banners, used for character counts.
The supplied Nox Orion 1.4 Ultra grades reproduce a raw score of 46.381 across 1,000 tasks. Arithmetic was checked against official rubric weights; answer quality was not independently regraded.
Nox Orion 1.4 Ultra (Nox): 46.381 — Self-test 2026-09-05; post-hoc adjusted 30.792. Arithmetic checked only. 3988.38 mean full-response characters.HealthBench is an OpenAI benchmark of 5,000 multi-turn health conversations graded against physician-written rubrics. HealthBench Hard is a 1,000-example subset selected because frontier models found those conversations difficult.
Rubric criteria reward accurate, context-aware and appropriately cautious responses. Recent reports may apply a length adjustment that lowers scores for longer answers and raises scores for shorter answers relative to 2,000 characters; length-adjusted and raw Hard scores are not comparable.
Default cohort: Meta Muse Spark eval methodology (Undated; checked Oct 24, 2026; last checked Oct 24, 2026). Cohorts are never combined and nothing updates automatically. HealthBench Hard and model names belong to their respective owners; REAGEN, Inc. is not affiliated with them.
Related: HealthBench announcement
Have a sourced HealthBench Hard result not listed here? Apply to the index.