The same model attempts the same repair five times and succeeds once. It is tempting to say it has learned the task, yet the next user gets one attempt and may encounter one of the other four paths. A success shows that a path can be found. Entrusting the task to the model requires it to find useful paths often. Much of capability development happens between these two states.
A successful trajectory often contains several well-placed choices: reading the right file, noticing an unobtrusive condition, changing approach after a failure, and performing an effective final check. The model can compose these actions without consistently composing them this way. Improving reliability means making useful choices more likely in their relevant situations and making those choices form complete stretches of work more often.
More attempts and better future attempts are different gains
The direct approach is to try more times and use a reliable check to select a success. The example below assumes independent attempts with the same success probability and accurate recognition of success. Increasing the number of attempts raises the chance of finding at least one success in this batch; the model’s success rate on an individual attempt stays unchanged. Raise that rate as well and compare the two numbers to see the change training aims to produce.
Batch probability = 1 − (1 − p)ⁿ. Here p is the rate per attempt and n is the attempt count.
Supervised fine-tuning offers a direct route: turn effective behavior into demonstrations that make their continuations more likely in similar contexts. People can write demonstrations, or they can come from selected model-generated trajectories. A final answer mainly demonstrates the shape of an answer. Keeping key observations, tool actions, and results also exposes the process of doing the work to training. Demonstrations grounded in situations the model will actually encounter make learned choices more useful.
Reinforcement learning lets the model discover practices that demonstrations have not fully specified. Give it tasks and an executable environment, let it act, and reward the results; the training algorithm adjusts its behavioral tendencies accordingly. A valuable condition is an environment that distinguishes attempts which actually solve the problem. The early DeepSeek-R1 report shows reinforcement learning improving reasoning on verifiable tasks and describes a training process combining demonstrations, reinforcement learning, and selected data. Demonstration supplies paths to learn, while exploration continues searching for better ones.
If the model is unfamiliar with the task’s objects and tools, repeated attempts alone can be expensive. Domain material and interaction records can first develop familiarity with the working environment. Human guidance and external workflows help complete steps the model cannot yet organize independently. Training then gains useful stretches of work to learn from instead of facing mostly unsuccessful attempts. Pretraining, mid-training, demonstrations, and rewards each shape the conditions under which useful behavior can be found and retained.
What counts as success shapes what gets learned
Evaluation criteria therefore help shape capability. If tests omit a boundary condition, the model may become good at passing those tests while continuing to make that error. If a score rewards only the appearance of a complete report, a jargon-filled report may be selected as a good demonstration. Every selection decides which behaviors deserve to become more common. Defining tasks, constructing environments, and designing evaluation belong to training alongside parameter updates.
The desired change must extend beyond the selected examples. After fixing the original problem, will the model still seek evidence and revise hypotheses in a different project with a different error? Development needs fresh tasks to test learned methods and known failures to guard later improvements. Training material must remain separate from held-out evaluation tasks, or familiarity with the questions can be mistaken for mastery of the method.
A product brings the process into a broader distribution of tasks. Real use encounters combinations of files, requirements, and environments that developers did not anticipate, exposing new capability gaps. Reconstructing a failure, identifying whether knowledge, method, or execution conditions were missing, and improving external support can produce better trajectories. Suitable learning material is prepared for training, while independent evaluation identifies where the new model improved. The data flywheel depends on keeping these pieces connected.
Seen this way, agent engineering continually participates in developing model capability. External systems help complete work that is not yet reliable; execution records turn effective practices into learning material; training makes the model more likely to make those choices independently. As the model improves, engineering moves toward new tasks. A further question follows: if improving working conditions helps models advance, can models begin improving their own working conditions?