October 2026
Some of the most striking results in machine learning came from systems that improved by interacting with something that pushed back. AlphaGo Zero learned Go from self-play alone, starting from random play with no human games, and beat the version of AlphaGo that had defeated a world champion by 100 games to 0 [1]. DeepSeek-R1-Zero was trained with reinforcement learning against simple rule-based rewards that checked whether its answers were correct and properly formatted, and its pass rate on the 2024 AIME math competition rose from 15.6 to 71.0 percent over training [2].
Both are loops. The model acts, something outside the model judges the result, and the judgment becomes training signal for a better model, which then acts again. When the loop works, each version of the model produces the experience that trains the next. That is what recursive self-improvement looks like in practice, and in every case where it has worked, the thing outside the model did much of the work.
It helps to name the parts. There is a policy: the model being trained. There is an environment, which decides what the policy faces and how the world responds to what it does. And there is a verifier, which decides how good the result was.
Go supplies the environment and the verifier for free. The rules are the environment, the opponent is a copy of the policy, and the winner is unambiguous. Many math problems have final answers that can be checked, and code often comes with tests that can be run, though neither check is perfect. Medicine does not supply a cheap, faithful simulator or a reliable verifier. There is no opponent, no rulebook that simulates a patient, and no cheap check that a decision was correct. Real outcomes arrive late, are tangled up with everything else that happened, and cannot be collected by letting a model practice on people. If self-improvement is going to work in medicine, both have to be built.
A fixed set of problems eventually runs out of signal. Problems that are too easy teach nothing, and so do problems that are impossible. Research on open-ended learning treats the environment as something to generate alongside the agent. POET pairs agents with environments and keeps creating new, harder ones as old ones are solved [3]. PAIRED trains an adversary to design environments, and uses a second agent’s performance to favor ones that are solvable yet still hard for the learner [4].
The clinical version of this idea is natural. The cases a model gets wrong are the ones worth more practice: the same disease presenting atypically, in an older patient, with a misleading first test, alongside a second illness. A self-improving clinical loop uses the policy’s failures to decide what it practices next.
In every such loop, the verifier limits what can be learned. A weak or exploitable verifier slows progress and steers optimization away from the intended task. If the verifier is wrong in some systematic way, the loop can amplify the error, because each generation is trained on what the last one was rewarded for.
Expert iteration and the Self-Taught Reasoner make this dependence explicit. Expert iteration alternates between a slow search that finds better moves and a fast network that learns to imitate them [5]. In the Self-Taught Reasoner, a model writes out its reasoning, keeps the reasoning that reached correct answers, retries the problems it missed with the correct answer as a hint, fine-tunes on the reasoning that works, and repeats [6]. Both work because the result can be checked. Swap in a checker that is right only most of the time, and the model learns the checker’s mistakes along with everything else.
For medicine, this makes the grader the most important component in the loop. It has to judge each decision against an expert reference, account for timing, and credit good process even when the outcome is unlucky, because whatever it gets wrong, the loop will learn and then reinforce.
The loop has more than one moving part. The policy improves against the environment. The environment’s supply of cases improves by finding where the policy fails. The grader has to hold up against policies that are, in effect, searching for ways to score without doing the work. Each part puts pressure on the others, and progress comes from the interplay. This is the same structure that made self-play work in games: no single component is good enough on its own, but each keeps the others honest.
Recursive self-improvement is usually discussed as a property of models. In practice it is a property of the loop a model sits in, and the loop is only as good as its environment and its verifier. Medicine is one of the domains where that loop matters most and exists least. Building it means building the parts medicine lacks: patients who respond to what the agent does, a grader that pays for good medicine, and a supply of cases at the edge of what the model can handle. That is the research problem we work on.