Put What the Model Cannot Yet Do into the Harness

A model can already write code, read tables, and draw charts. Ask it why a production line behaved abnormally last night, and it may misalign two devices’ timelines, mistake scheduled maintenance for a fault, and produce an elegant report around those errors. It has many general capabilities, while this particular job still needs several things: how to interpret local data, how experienced people investigate it, and which operations produce reliable evidence.

The most immediate repair is to put equipment descriptions, maintenance logs, and diagnostic advice into the prompt. This works with a small amount of material. As tasks accumulate, however, the prompt becomes an ever-growing manual. Every investigation carries the background of every machine, obscuring the conditions relevant now. Knowledge bases, long-term memory, workflows, and specialized tools emerge as ways to organize different missing parts of the job.

What does each existing approach supply?

Knowledge bases and retrieval address the problem of having too much material to read every time. The model can find documents by equipment, symptom, and time, recovering units, maintenance schedules, and past records. This supplies evidence without necessarily supplying a method for using it. Finding a calibration record still leaves the need to notice that its timing coincides with a step change in a curve. The facts may enter the window while the diagnostic method remains absent.

Long-term memory preserves previous work. A past misdiagnosis and the evidence that corrected it may help the next investigation. Storing complete conversations, however, also preserves tentative ideas, mistakes, and accidental successes. The next task needs the case’s applicability conditions and decisive turn: which symptom should bring it to mind, and which verification changed the conclusion? Memory often becomes useful when an episode is distilled into transferable experience.

Workflows move experience toward execution: inspect data, locate anomalies, then verify explanations. They prevent omissions and coordinate roles in a known order. As conditions vary, branches multiply: does this machine have calibration records, was that sensor replaced, does this anomaly need another route? A workflow that tries to own every judgment approaches a reimplementation of the business program. Explicit dependencies and mandatory conditions are good candidates for fixed structure; situational judgment needs room to remain with the model.

Specialized tools make particular actions dependable. Time alignment, missing-value handling, and unit conversion can be delegated to verified programs. A growing tool collection introduces another selection problem: which tool fits this situation, what limits apply to its result, and what should be checked after failure? A list of tool names lacks working context. Knowledge bases, memory, workflows, and tools are all useful; what is often missing is a way to organize them around the same job.

Four kinds of support that a job can accumulate

• Knowledge preserves facts and relationships. Field units, device identifiers, timestamp time zones, and the times of last night’s shutdowns all belong here. Knowing how vibration works cannot reveal which field this factory labels “amplitude.” Such information needs a retrievable source that stays current as conditions change. Documentation and records let the model apply its general understanding to the actual objects in front of it.

• Experience preserves situations and judgments. An engineer may know to check calibration records before suspecting a machine fault when a sensor trace jumps after calibration. Experience connects clues with actions: what to inspect first after a particular observation, and what evidence rules out an explanation. Preserving the conditions and outcome makes an episode useful in a later investigation. Reducing it to “sudden jumps are usually fine” turns a useful judgment into misleading advice.

• Workflows preserve methods and dependencies. Establish whether the data is usable, locate the abnormal interval, compare explanations, and deliver conclusions with their evidence. A workflow preserves dependencies between pieces of work. A model may know how to perform every step yet omit one during a long task; the workflow can initially maintain those relationships for it. The system can check mandatory delivery conditions while leaving other steps as guidance the model adapts to the situation.

• Tools preserve practiced actions. Knowing that two traces need alignment still leaves missing values, sampling intervals, and clock offsets to handle. The model can write a program on demand or call a validated processing function. The latter turns a recurring precise operation into a stable interface, avoiding repeated implementation and defining its inputs, outputs, and errors. A specialized agent can thereby draw on existing software capabilities without recreating them from text each time.

The example below lets you add these four forms of support individually. The model and task stay the same; what changes is the evidence and assistance available for doing the work. Notice the blind spot each addition addresses. Enabling all four supplies the conditions shown here; the eventual report still has to stand up to the data and business facts.

Why do these look like forms of human memory?

Knowledge, experience, and workflows have familiar counterparts in cognitive science: semantic memory holds facts about the world, episodic memory holds experiences in specific circumstances, and procedural memory supports learned skills and practices. Squire and Dede’s account of memory systems discusses these different functions. For engineering, this suggests that a system that works over time needs to know how the world is, remember what it has encountered, and accumulate ways of doing things. A searchable knowledge base alone cannot cover all of those needs.

Where do tools fit? I like to think of them as the system’s “muscle memory.” A person learning an action attends to its steps; with practice, many details run together smoothly. An agent can similarly turn precise operations it repeatedly derives or generates into programs it can call next time. Cognitive science generally places such motor skills on the procedural side of memory. Engineering lets us further separate readable methods from executable actions: a workflow guides the organization of work, while a tool directly performs part of it.

These forms of support can also transform into one another. An investigation leaves an episode; we distill the lesson to check calibration when a curve jumps, then incorporate it into a diagnostic workflow. Repeated time alignment becomes a tool, whose results add new facts and cases. A method of working can therefore become more practiced outside the model without waiting for every detail to enter its parameters.

Organize the material in a skill folder

Seen from here, Skills fit these needs closely. A natural-language entry point can describe the task, method, and applicability conditions; documents, cases, templates, and scripts can sit alongside it. A production-line diagnostic skill explains when to use it and what to establish first, with references for field meanings, cases for recurring misdiagnoses, and scripts for precise processing. The things a job needs acquire a shared entry point while retaining forms suited to their individual roles.

Progressive loading in Agent Skills keeps that entry point light. The model first sees names and descriptions, reads the selected skill’s instructions, and then uses references and programs as needed. A large skill library does not require unfolding every job in the current window. Descriptions help find a method, instructions guide its use, and external files carry details. Index and content layers follow naturally from how the material is used.

For example, the skill can use an ordinary directory structure:

line-diagnosis/
├── SKILL.md                  # Applicability, method, material references
├── references/fields.md      # Field meanings, units, and data sources
├── cases/calibration.md      # Symptoms, investigation, and conditions
└── scripts/align_series.py   # Time alignment and result validation

After reading the entry point, the model can choose material and actions for the situation. Guidance and executable checks can be adjusted independently.

This organization retains the strengths of the earlier approaches. A knowledge base can still handle retrieval, historical records can retain original episodes, a workflow can enforce fixed dependencies, and specialized tools can perform calculations. The skill provides a readable, editable account of how this job should use them. It is especially suitable for methods still evolving: people can edit them, models can read them, and completing a task can suggest the next revision.

Using a folder still requires maintaining its instructions, cases, and scripts. Vague descriptions cause poor selection, cases retaining only conclusions invite misapplication, and broken script dependencies prevent execution. Locate the gap before editing: an interface omitting calibration records needs an interface change; acceptance that permits an unsupported report needs better checks; a model lacking foundations for understanding vibration needs relevant knowledge and training. Skills suit the accumulation of working material and methods. Evidence from the task determines which part actually needs to change.

Along this path, the harness becomes a place where working methods can grow first. A reminder to check calibration records can become experience the model retrieves proactively; later training may teach it to look for relevant environmental changes whenever similar symptoms appear. A case becomes experience, experience is organized into a skill, the skill helps produce better traces, and those traces participate in training. What attracts me to Skills is that improvements can be written into cases, instructions, and scripts, then used in the next job.