HealthBench AIReading HealthBench Professional results
Our guide to clinical conversation scores, length adjustment and why evaluation protocols matter.
By HealthBench AI · REAGEN, Inc. ·
Our editorial note. This page is written by HealthBench AI. Research findings and submitted results belong to their cited sources; this is our explanation, not a republication of their articles or an endorsement by their authors.
What the benchmark measures
HealthBench Professional is a separate set of 525 physician-authored tasks covering care consults, writing and documentation, and medical research. Responses are evaluated against task-specific, weighted rubrics. It is not a multiple-choice exam, and its scores should not be interpreted as the percentage of patients a model would treat correctly.
Artificial Analysis has published its own evaluation of this benchmark. Our index attributes those results to Artificial Analysis rather than presenting them as tests performed by HealthBench AI.
Why response length matters
A longer response has more opportunities to satisfy rubric items, but verbosity is not the same as clinical quality. The documented Professional adjustment centers response length at 2,000 characters: adjusted percentage points = raw percentage points − 1.47 × (characters − 2,000) / 500. Shorter responses receive the corresponding adjustment in the other direction.
Raw and adjusted scores are different metrics. Judge models, inference settings, repetitions and tools can also differ between evaluators. We preserve those distinctions instead of treating every published number as a controlled head-to-head result.
Our own evaluation
HealthBench AI also operates a separately identified, two-model Professional evaluation. Both models use the same API harness with no search or external tools, while their model-specific reasoning efforts remain disclosed. The live run report records the grader, budget, progress, exceptions and evidence hashes.
Only complete results replace the two model versions’ current source-directory entries. Original publisher reports remain accessible as source history. Our run is not OpenAI certification and does not reproduce the publisher’s unpublished internal implementation.
Sources and supporting evidence
Explore the benchmark results · Read our methodology · Suggest a correction