The same model attempts the same repair five times and succeeds once. It is tempting to say it has learned the task, yet the next user gets one attempt and may encounter one of the other four paths. A success shows that a path can be found. Entrusting the task to the model requires it to find useful paths often. Much of capability development happens between these two states.
A successful trajectory often contains several well-placed choices: reading the right file, noticing an unobtrusive condition, changing approach after a failure, and performing an effective final check. The model can compose these actions without consistently composing them this way. Improving reliability means making useful choices more likely in their relevant situations and making those choices form complete stretches of work more often.
More attempts and better future attempts are different gains
The direct approach is to try more times and use a reliable check to select a success. The example below assumes independent attempts with the same success probability and accurate recognition of success. Increasing the number of attempts raises the chance of finding at least one success in this batch; the model’s success rate on an individual attempt stays unchanged. Raise that rate as well and compare the two numbers to see the change training aims to produce.
Batch probability = 1 − (1 − p)ⁿ. Here p is the rate per attempt and n is the attempt count.
For example, a 20% per-attempt success rate gives about a 67% chance of at least one success in five independent attempts. If success is recognizable and failures are affordable, extra attempts can already improve a product. But give five users one attempt each, and the chance that all five succeed is only 0.032%. Agent evaluations distinguish at-least-one success from success on every trial. A demonstration can select a good result; dependable service needs better odds on each delivery.
How do we improve those odds? Four connected jobs help explain the process: make good paths discoverable, establish whether they really succeed, teach the model the choices they contain, and test whether the change transfers to new tasks. Training algorithms participate in the third job; tasks, environments, and evaluation determine what there is to learn. Understanding that system is more useful than attributing every improvement to more data and stronger training.
First, make useful practices appear
Imagine a model fixing a configuration issue. It often edits a generated file directly; an immediate test passes, but the next build overwrites the change. An experienced person asks what generates the file, finds the source configuration, changes that source, and rebuilds before verifying. A single crucial query may separate the two attempts, sending the rest of the work down different paths. The model needs to learn when that query is worth making.
A person can demonstrate this path, a skill can prompt an investigation of generation relationships, a tool can expose dependencies, and a larger attempt budget may let the model discover the route itself. These supports change the conditions of exploration so useful behavior is not drowned in ineffective attempts. Domain material and mid-training help from another direction: familiarity with generated files, build systems, and dependencies makes it easier to turn observations into useful judgments.
External support can contribute this way: a person need not write every answer; supplying the gap that blocks progress may let the model connect the rest. A stronger model can provide demonstrations, existing software can provide exact calculations, and an environment can provide immediate feedback. Capabilities combine and develop under those conditions, and exploration can produce new practices. Completed work becomes the source of learning material.
Then establish what success actually means
If acceptance checks only the edited file’s contents, the wrong path can receive full marks. Rebuilding exposes the problem; checking previously working behavior reveals regressions. Evaluation therefore participates in defining the capability. What “the model can fix configuration issues” means depends substantially on the conditions it faces and the consequences that are checked.
-
Does the outcome hold? Check the target artifact, actual environment state, and required behavior. Execute builds, query state, and validate data where possible. Subjective quality needs an explicit rubric, with model-based judgments calibrated by people where necessary.
-
Does success respect the task’s constraints? Bypassing checks, damaging existing data, or quietly narrowing the request can produce a superficially good result. Evaluation should protect conditions that cannot be sacrificed while allowing different valid solutions. Requiring every demonstrated step would confine exploration to an old method.
-
Is the path worth reusing? Two attempts may both finish, while one investigates clearly and verifies appropriately and the other guesses until something passes. Time, call cost, unrelated modifications, and evidence for decisions help distinguish them. Reasonable recovery from failure is useful material too; the model needs to learn recovery as well as imitate ideal routes without mistakes.
A learnable trace should connect the task, observations actually available to the model, chosen actions, and environment results. Keeping only the final patch loses the situation that prompted searching for the generator. Placing a later answer into an earlier input teaches a student that will lack that information in real work. Trace preparation should preserve the conditions of important choices while removing erroneous records, sensitive data, and material unsuitable for reuse.
Training changes how the next choice is made
Supervised fine-tuning treats selected traces as demonstrations. The model learns subsequent content and actions in their contexts, and parameter updates make those choices more likely later. The configuration example can demonstrate noticing a generated-file marker, locating the generator, editing the source, and rebuilding. Beyond the changed line, this offers transferable relationships: the visible file may be an output, and a durable fix requires finding what produces it.
Reinforcement learning lets the model continue attempting tasks in an environment and adjust behavior through rewards. We can test whether a change survives rebuilding without prescribing one solution to every problem. The model can explore paths and increase the likelihood of choices associated with good outcomes. The early DeepSeek-R1 report presents reinforcement learning on verifiable tasks alongside a pipeline combining cold-start demonstrations, reinforcement learning, and selected data. Demonstrations provide useful starting points; exploration advances from them.
Establishing whether the configuration was repaired already required the original project, rebuild commands, and result checks. Restore the project and the model can try again, receiving feedback from those checks. During development, this setup compares versions of a tool or procedure; during training, it supports model exploration. The task environment begins to serve both kinds of improvement. SWE-Gym organizes software tasks, executable environments, and tests to make real software work repeatedly available for training.
- Prior training：Pretraining · mid-training · post-training
- Task and current context：Goals · facts · constraints
- Model：Interpret the situation; choose an action
- External support：Expert guidance · skills · existing tools
- Task environment and verifier：Reset · execute · observe · evaluate
- Supervised fine-tuning：Select useful trajectories; Learn effective choices in context
- Reinforcement learning：Explore within the environment; Use outcome feedback to adjust behavior
Useful traces provide demonstrations; reproducible environments and verifiers support further exploration. The updated model returns to tasks to test what it learned.
These activities change different things. A prompt or skill changes the conditions available now. Additional attempts and selection improve the chance of finding a good result in this run. Training updates parameters and changes behavior on subsequent tasks. They work together, but assessing a gain still requires knowing which layer improved.
New tasks reveal what was learned
Solving the configuration issue used in training is a starting point. Will the model still investigate source relationships with a different build system, a less visible generation marker, or another file type? Will it overinvestigate tasks that need no such inquiry? Evaluation should test both where a behavior belongs and where it does not. That distinguishes learning a method from acquiring another mechanical habit.
Failures discovered during development can become regression checks protecting acquired capability. Judging generalization also requires new tasks untouched by training and repeated tuning. Comparisons should keep environments, tools, and attempt budgets comparable; otherwise, higher scores may come from stronger external assistance. We need to know both whether the product improved and whether the model can work more independently. Both are valuable, and both should be measured clearly.
Products close the loop
Real use keeps introducing unfamiliar project structures, incompletely expressed needs, and failures specific to an environment. Reconstructing an issue as a repeatable task, adding it to evaluation, and identifying changes to knowledge, methods, tools, or execution conditions turns a production failure into a basis for development. Meanwhile, expert guidance and improved systems produce more useful traces for verification and training. Failures reveal the next gap; successes supply learnable paths. The two loops reinforce one another through the same work.
A stronger model returns to these tasks, potentially needing fewer external reminders and handling more complex situations. Tools and environments keep running, evaluation keeps checking, and new problems drive another round of improvement. This is how an isolated success becomes a capability that can be delivered reliably. Self-improvement then becomes a concrete question: can the model help diagnose failures and improve tools and skills, thereby improving the conditions of its next round of learning?