How Do Real Needs Become a Direction for Agent Improvement?

The appeal of recursive self-improvement is that an agent can take on the work of improving itself. It finds a weakness, changes its methods, and uses those better methods to find the next weakness. Once moving, the loop seems capable of accelerating. Before the first edit, though, it needs a direction: what change counts as an improvement?

Writing a longer reflection, adding a planning step, or adopting more sophisticated instructions is easy. The user may simply want a spreadsheet cleaned up, its formulas preserved, and another sheet left intact. The direction of self-improvement has to grow out of those real requirements. Work must become a task that can restart, permit different approaches, and still have a judgeable outcome. An evaluation set takes on a more interesting role here: it defines the kind of working life the system is preparing to learn.

Putting a real computer inside the test

OSWorld takes a direct approach: agents use real operating systems and applications. LibreOffice handles spreadsheets, Chrome handles websites, and files live in a filesystem. The agent sees the screen and operates the mouse and keyboard, changing state actually held by the software. Wrong windows, missing menus, and content lost while copying between applications can all occur naturally. The software already implements much of the complexity that researchers need to capture.

A subtler detail is that work often starts halfway through. Users rarely hand over an empty, freshly booted computer: a document is open, the browser is on some page, and files remain from earlier work. OSWorld prepares these situations through task configurations that download material, launch applications, and arrange windows. If every task started from a clean home screen, an agent’s apparent fluency might amount to knowing the entrance to the test.

Virtual machines make repeated trials possible. The system restores a base snapshot, then constructs the task situation from its configuration. After a run, it can discard the changes and start again. Combining snapshots with initialization scripts also avoids storing an enormous machine state for every individual task. The same failure can then be attempted with an old method, a new method, or another model. A work episode that once happened only once becomes an object of controlled comparison.

134 functions asking whether the work is done

The original OSWorld paper pairs 369 tasks with 134 distinct evaluation functions. That number is revealing because it exposes the highly specific work behind a general-purpose agent. Determining whether a spreadsheet was edited correctly and whether particular browser data was deleted requires understanding different objects. The model can be shared, as can the mouse and keyboard. What completion means still has to be established task by task.

Its evaluation separates evidence collection from judgment: retrieve relevant state, then apply the appropriate rule. Examples in the paper include reading Chrome cookie data to check removal of a site’s records, retrieving an edited spreadsheet for comparison, and inspecting an accessibility tree to check the current interface. The outcome may live in a database, a file, or a screen. A verifier has its own observation interfaces; it need not be restricted to the screenshot available to the agent.

Consider a further spreadsheet example. For a request to display amounts with two decimal places, the relevant property is the cells’ number format. Two zeros happening to appear in a screenshot do not establish that formatting is correct; text can produce the same appearance. If later calculation is required, numeric types matter too. OSWorld’s spreadsheet evaluator opens workbooks and checks configured cell, style, and data-validation properties. Inspecting the object itself makes it possible to distinguish an apparently correct result from one that meets the requirement.

This form of acceptance gives solutions room to vary. Menus, shortcuts, and scripts may all produce a correct file; any route within the task’s permitted action space can qualify. We specify the necessary properties of the outcome and leave intermediate choices to the agent. Conversely, if a user requires an editable artifact but the verifier checks only an exported image, the whole system is drawn toward an easy mistake: delivering attractive screenshots while losing what the user actually needs.

What must be controlled to reproduce a world?

A desktop snapshot controls the desktop. It cannot undo an order already submitted to a remote service or rewind data on an external website. WebArena therefore deploys shopping, forum, and code-collaboration sites in controlled environments with ways to restore their initial state. It retains website navigation, forms, and business relationships while bringing backend state into the experiment. Whether a world can restart depends on who controls its consequential state.

SWE-Gym applies the same idea to repositories: a natural-language issue, its codebase, an executable environment, and tests form a task together. An error report alone lets a model discuss causes. Adding dependencies and tests lets it edit, run, observe, and try another approach. Desktops, websites, and repositories look very different, yet share one achievement: turning a one-off work episode into an experience that can recur and provide feedback.

For a new domain, I would establish four things before deciding how many tasks to collect:

• Where work begins. Inputs, existing progress, account state, and external dependencies determine the difficulty actually encountered.
• Which choices affect the outcome. Observations must support decisions and actions must cover the work. Crucial dependencies need reconstruction or substitutes with credible behavior.
• What evidence completion leaves. Artifacts, business records, state changes, and quality requirements must line up, including properties that must remain intact.
• Which related situations will occur. Varying inputs, starting states, task combinations, and exceptional conditions reveals where a method applies.

For example, cleaning a customer list may involve deduplication, field repair, and export. A useful practice environment also includes duplicate names, missing fields, existing filters, and columns referenced by formulas. We do not need to reproduce an entire company; we need the causal relationships that affect this work. If deleting a column really breaks references, the agent has a reason to check first. If the simulator omits that consequence, the resulting fluency has a gap. The most valuable realism is that actions carry the consequences they would have in practice.

Evaluation becomes part of development

A restartable task set changes how we modify an agent. Run the current system and observe where it gets stuck; add domain material, improve a tool interface, or revise a skill; then compare from equivalent starting points. The agent is not always what needs changing. Ambiguous requests, stale data, and faulty verifiers also produce failures. The first round of evaluation development often teaches us to understand the business more clearly.

Three outcomes are easy to conflate: the original task now passes, a family of tasks passes more often, and real usage improves. The first establishes a concrete repair. The second requires varying inputs and starting points. The third also depends on how well the task distribution represents users’ work, and whether cost, speed, and consequences are acceptable. Repeatedly editing against one task can make progress, but the method we retain should transfer. Tasks used to select changes should be separated from those used for final assessment.

After deployment, the task collection can keep growing. A user correction can become a new scenario; a failed run can reveal an overlooked starting condition. This requires enough evidence to reconstruct the situation. Saving only the words “export failed” leaves little to investigate. Preserving the input file, actions, software version, and actual artifact makes it possible to locate where the work went wrong.

A testing room can also be a practice room

Look at the infrastructure again: tasks, an environment for action, resets, and outcome feedback. These are already the basic conditions needed for repeated exploration during training. Useful runs can become demonstrations for supervised fine-tuning. The environment can also support reinforcement learning, allowing different actions to be tried and choices adjusted through feedback. Infrastructure built to compare two harnesses remains useful when work moves into training.

Being scoreable does not automatically make a task learnable. If success arrives only after a hundred steps, a model may rarely encounter behavior worth learning from. Human demonstrations, existing tools, skill guidance, and progressively harder tasks can help it reach useful paths. Evaluation establishes the destination; the surrounding support makes that destination reachable. Together, they turn a difficult business requirement into a direction that can be developed and explored.

This is why I put evaluation at the beginning of self-improvement. The idea of an agent editing itself is compelling enough to make us build the editor first. Yet what that editor can accomplish depends on the world it can observe and test. Making a new domain reproducible, and grounding user requirements in credible verifiers, creates a new source of experience for intelligence. The next question becomes more concrete: after those experiences, what improvements should the agent retain?