HealthBench AINox Orion: our test results
We tested Nox Orion and obtained these scores: 69.53 on HealthBench Professional and 30.79 on HealthBench Hard, both length-adjusted.
By HealthBench AI · REAGEN, Inc. ·
Tested by HealthBench AI. We tested Nox Orion and obtained these scores. This report presents our results, testing setup and supporting evidence.
We tested Nox Orion
We tested Nox Orion and obtained these scores. HealthBench AI ran Nox Orion 2.1 on HealthBench Professional and Nox Orion 1.4 Ultra on HealthBench Hard. The results below come from our model evaluations.
Nox Orion 2.1 scored 69.53 length-adjusted on HealthBench Professional. Nox Orion 1.4 Ultra scored 30.79 length-adjusted on HealthBench Hard. The raw scores were 72.83 and 46.38, respectively.
We evaluated Nox Orion 2.1 using a tool-assisted, multi-stage setup. We retain the run records and provide the score calculations, confidence intervals and methodological limitations below.
Orion 1.4 Ultra: our length-adjusted result, not the Nox website score
Our HealthBench Hard length-adjusted score for Nox Orion 1.4 Ultra is 30.79. This is the result we report from our test records after length adjustment; it is not the score published at getnox.ai/models.
The Nox models page publishes 46.4% for its Nox 1.4 Orion run, with Level 5 Ultra covering all 1,000 Hard examples. That page does not label its 46.4% figure as length-adjusted. Our raw result is 46.38, which also rounds to 46.4; our length-adjusted result is 30.79.
The difference between 46.4 and 30.79 should not be read as a like-for-like drop in model performance. They are different metrics. We applied the documented Hard correction to the full recorded response text: raw percentage points − 3.92 × (characters − 2,000) / 500. We attribute 30.79 to HealthBench AI, not to Nox’s website.
Orion 2.1 remains around the same score
Nox’s Orion 2.1 page publishes a HealthBench Professional length-adjusted score of 69.5 across 525 tasks. Our test result is 69.53, approximately 0.03 points away and the same 69.5 when rounded to one decimal place. Orion 2.1 therefore remains around the same score in our reported results.
Unlike the Hard comparison above, these two Professional figures are both labelled length-adjusted. The numerical agreement does not by itself establish that every run setting was identical. Our test configuration, development-task exposure and uncertainty remain disclosed below.
Professional results and uncertainty
Our 525-task Professional evaluation records give a recomputed raw mean of 72.83 and a length-adjusted mean of 69.53. The adjusted task-bootstrap 95% interval is approximately 65.81–73.03. This interval comes from resampling tasks; it is not a guarantee about repeated model runs or grader accuracy.
Our run records disclose 40 development tasks and 485 tasks declared held out. Those counts describe our evaluation’s exposure, not an official split of the public Professional benchmark. The declared-held-out adjusted mean is approximately 69.84, but the exposure disclosure remains relevant to interpretation.
Hard results and supporting evidence
For our Nox Orion 1.4 Ultra Hard run, the recomputed raw mean is approximately 46.38. Applying the documented Hard length correction to the full recorded response text gives approximately 30.79, with a task-bootstrap interval of 28.97–32.58. Hard and Professional use different correction coefficients.
The machine-readable audit includes input hashes, formulas and aggregate results. It does not certify that every rubric verdict is correct, prove clinical safety or establish a fair comparison against models evaluated under different tools and protocols.
Questions about this evaluation
What did Nox Orion 2.1 score on HealthBench Professional?
HealthBenchAI reports 69.53 length-adjusted points and 72.83 raw points across all 525 tasks, graded with GPT-5.4 at low reasoning effort. These results apply to the tested product configuration; the methodology and comparison limits are described below.
Were all 525 Nox Orion 2.1 tasks held out of development?
No. The package declares 40 previously used development tasks and 485 tasks held out of tuning. The declared holdout subset scored 69.84 length-adjusted points. This is a disclosed split, not proof of zero training-data contamination.
Can the Nox evidence package independently verify every answer?
No. The Professional package contains per-task scores and character counts, allowing aggregate arithmetic to be checked, but not independent regrading of the answers. The re-uploaded per-example file matches the previously reviewed SHA-256: dd2be140724809dd73791e1af84d4264dee71bf3b1adf732ca007d1805d1a082.
Is this a controlled comparison with the GPT and Claude API tests?
No. The Nox product run used live web evidence and a multi-stage workflow. Our GPT and Claude API tests used no web search or external tools. Numerical score differences do not establish statistically significant superiority, clinical safety or an official cross-provider rank.
Sources and supporting evidence
Explore the benchmark results · Read our methodology · Suggest a correction