HealthBench AIOrion 2.1 Pro: HealthBenchAI score verification and independence
REAGEN, Inc.’s HealthBenchAI reports 69.53 length-adjusted points for Orion 2.1 Pro across 525 Professional tasks. Results, verification scope and our no-affiliation statement.
By HealthBench AI · REAGEN, Inc. ·
Tested by HealthBench AI. We tested Nox Orion and obtained these scores. This report presents our results, testing setup and supporting evidence.
Who evaluated and verified this result?
HealthBenchAI is operated and published by REAGEN, Inc. We are not affiliated with, funded, sponsored, endorsed or supported by Nox AI, xHealth or any AI company. Product names and logos identify the systems evaluated, not partners or sponsors. This is our operator declaration, not an endorsement from a model developer.
The HealthBenchAI operator reports personally running the original evaluation and verifying the recorded scores. “Third-party score verification by HealthBenchAI” means verification by an evaluator separate from Nox AI and xHealth. It does not mean another organization certified HealthBenchAI, that Nox endorsed the result, or that OpenAI certified the evaluation.
What score did Orion 2.1 Pro receive?
The existing 525-task HealthBench Professional run scored 69.5277657684534 length-adjusted points (69.53 rounded) and 72.83285776845341 raw points (72.83 rounded). The 95% task-bootstrap interval for the adjusted mean is 65.81–73.03. This interval measures task-sampling uncertainty, not all grader or run-to-run uncertainty.
The run includes 40 previously used development tasks and 485 tasks declared held out of tuning. The declared holdout subset scored 69.84 adjusted points. Three failed tasks were recovered. Development exposure, failure recovery and the single-run design remain part of the result’s interpretation.
Why does the retained evidence say Orion 2.1?
At the operator’s request, the leaderboard now displays this entry as Nox Orion 2.1 Pro. The original package and evaluated configuration were identified as Orion 2.1. This is a display-name update to the existing result, not a new evaluation, a new score, or independent proof that a differently deployed Pro snapshot behaves identically.
The original run date remains October 23, 2026. This article was published on November 5, 2026. Evidence hashes and original numerical values are preserved; the name update does not alter the underlying evidence.
What exactly was verified?
The later evidence review recomputed score arithmetic from retained per-task raw scores, adjusted scores and character counts. Professional length correction is raw score − 1.47 × (visible characters − 2000) / 500; task values are not clipped before averaging. The aggregate is bounded to the reported score range.
The per-example evidence SHA-256 is dd2be140724809dd73791e1af84d4264dee71bf3b1adf732ca007d1805d1a082. The published JSON identifies the evaluator, display-name history, score summary and verification limits. Hashes establish file identity, not proof that every answer or grader verdict is correct.
The package supplied for the later arithmetic review did not contain Professional answer transcripts or criterion-level grader records. That review did not perform fresh model inference or independent answer regrading. Verification of arithmetic is therefore narrower than independent answer-level replication or clinical validation.
What testing conditions limit comparison?
The original package documents a tool-assisted product configuration with planning, live web evidence, drafting, audit and revision, graded with GPT-5.4 at low reasoning effort. That configuration differs from our GPT and Claude no-web API tests and from published multi-repeat evaluations.
Numerical placement is not a controlled head-to-head result, proof of statistically significant superiority, an official benchmark rank, medical advice or clinical safety certification. Raw and length-adjusted results must not be interchanged.
Sources and supporting evidence
Explore the benchmark results · Read our methodology · Suggest a correction