FOR FRONTIER LABS

The training environment for medical AI.

They pass the boards and fail the patient. Train and grade frontier models in a stateful hospital.

Request access See the leaderboard →
An exam grid dissolving into a living hospital ward

THE THESIS

Medicine isn't a question with an answer.
It's a decision with a consequence.

THE PLATFORM

An environment, a verifier, and the data to train on.

Run a model. Grade every action. Train on what comes back. One loop.

CASE 4471 · 88F · RUPTURED AAA
HR128▲ rising
BP70/40▼ falling
SpO₂88▼ falling
T+04order CBC, BMP
T+31review labs
T+88reassure, observe
T+137ARREST
STEP 01 RUN
Train against a real hospital
60+ turn episodes on a locked seed. The patient deteriorates while the model decides.
60+ TURNS · LOCKED SEED
PendingVerified
PHYSICIAN ANSWER KEY · CASE 4471
✓T+04order CBC, BMP+2
✓T+31review labs+1
✗T+31no type & cross−6
✗T+88didn't escalate−9
✗T+137missed arrest−18
⊘ order-farming · blockedmodel −47 · md +105
STEP 02 GRADE
Grade with a physician-anchored judge
A dense, per-action reward from physician answer keys, hardened so models can't game it.
8 DIMENSIONS · 17,000+ TESTS
CASE 4471 · 88F · RETRY ON SAME SEED
before✕ died T+169
after● survives
trained model escalates @ T+90 · transfuse · admit
base 39 → trained 57 · +18 · held-out gold
STEP 03 TRAIN
Get data that moves models
Train on graded runs and an 8B model climbed +18 points toward a physician.
+18 PTS · HELD-OUT GOLD

THE LEADERBOARD

Zero of six reach a skilled physician.

Every frontier model scores below the physician on decision quality. The best reaches 65%.

Select a model for its skill breakdown% of a skilled physician · anchor = 100
01
Opus 4.8best of the lineup
65%
02
DeepSeek V4 Pro
61%
03
GPT-5.5
60%
04
Kimi K2.6
58%
05
Gemini 2.5 Pro
49%
06
Grok 4.3
21%

Decision quality, as a % of a skilled physician · Gold-30 benchmark · averaged across 3 seeds.

How it's scored →

THE LOOP

A patient with a clock.

One real case, one locked seed: a stable-looking 88-year-old hiding a ruptured aorta. You're the attending. Make the call.

CASE 4471 · 88F · TURN 12 / 60HIDDEN TRUTH · SEALED

Vague abdominal pain. BP 96/60, HR 92 — she looks well. She could be eleven things, and the truth stays hidden until the case ends. What do you order?

01 / PRESENT
A case, not a question
A disease, a patient, an answer key, locked on a seed so it replays exactly.
02 / ACT
The model acts, turn by turn
It scans the aorta (clean), calls it sepsis, waits on a CT, while the patient crashes.
03 / GRADE
Every action is graded
No escalation, no blood, no disposition. Reward −47 against the physician's +105.
04 / TRAIN
It trains and climbs
Train on graded runs and an 8B model climbs 39 → 52 → 57%. +18 points.

FOR THE LABS

A reward you can train on, and can't game.

Most reward functions can be gamed: optimize against them and the score climbs while the medicine gets worse. Ours is built to resist that. Every action is graded on its own, every shortcut is refused, and the same run lands on the same score, every time.

STATEFULthe patient mutated this turn DENSEgraded across 8 dimensions HARD TO GAME3 reward hacks blocked REPRODUCIBLEreplays to an identical score
episode aaa_88f_ruptured · seed 0x4F2A · turn 18 / 60
patient_state → BP 78/44 · HR 122 · lactate 4.1 · deteriorating
reward.decompose()
diagnosis+2.1
safety+3.0
workup+1.5
timing−0.8
communication+0.6
procedure+0.4
disposition0.0
stewardship−0.5
guards.applied[]  ·  reward hacks refused
order_every_test_farmingBLOCKED
look_alike_creditDENIED
outcome_decoupled_from_treatmentREJECTED
step_reward +6.3  ·  replay → identical  ·  17,000+ tests green

VALIDATION

Answer keys, generated by practicing attending physicians.

They author and tune the keys, and the cases are held out so no model can memorize the answers.

QUESTIONS

Questions, answered.

It runs as a standard RL environment. Episodes expose agent-callable tools, return new state and a per-action reward, and replay on a locked seed. You point your trainer at it and collect graded trajectories, the same way you would any environment.

Every action is scored against physician answer keys across eight reward dimensions, diagnosis, safety, procedure, timing, and more, with a lexicographic safety veto. Scores are reported as a percent of a skilled-physician scripted run.

We adversarially attack our own grader, patch the shortcuts (order-every-test farming, look-alike credit, outcome decoupled from treatment), and guard it with 17,000+ CI-gated tests. The reward is built to survive optimization.

Ground truth comes from practicing physicians, outcome baselines are grounded in real hospital data, and we hold out a frozen Gold set to measure transfer you never trained on. It is a training environment, not a deployed device.

Today it covers emergency and acute care, the highest-stakes decisions in medicine. The same engine extends across every specialty and care setting, and to every signal a clinician reads, from notes and labs to imaging, waveforms, and video, toward a full hospital world model.

Frontier labs and clinical-AI teams training or evaluating medical models. It is invite-only today. Request access and tell us what you are training.

ACCESS

Put your model in the hospital.

Invite-only for frontier labs and clinical-AI teams. Tell us what you're training.

Request access Read the methodology →