The Harness Assumes the Model Holds Still

The assumption underneath

Harness engineering treats the model as the fixed part of the system and the environment as the variable part. Data changes. Policies change. Tools time out. Users phrase things unexpectedly. The harness is the layer that absorbs all of that variance so the model can be relied on to do the one thing it does.

That is a reasonable assumption when the model is an API endpoint operated by someone else. It is a bad assumption in an open-weight deployment.

In an open-weight deployment, the model is a file on disk. It gets fine-tuned on institutional data. It gets quantized to fit the GPU budget. It gets merged with another checkpoint because someone found a recipe that scored well. It gets swapped for a newer release with the same name. Each of those transformations produces a new artifact whose behavior differs from the artifact that was evaluated, and none of them touch anything the harness can observe.

The permission boundary still holds. The tool schemas still validate. The traces still record faithfully. Every guardrail in the harness continues to function exactly as designed.

And the component in the middle answers differently than the one that was tested.

What a trace tells you and what it does not

The trace is the strongest idea in the harness argument. A record of the request, the context that was assembled, each tool call and its result, the permission check, the confirmation, the final response. When someone asks six months later why an account was changed, the trace is the starting point.

A trace answers the question "what happened." It is a record of a past event, and it is excellent at that.

It does not answer the question "what should this build do." For that you need a baseline: a set of adversarial probes run against this specific artifact, with recorded results, produced before the build went into production. Without that baseline, a trace showing a bad outcome cannot distinguish between three very different situations. The harness had a gap. The user found a novel input. Or the weights are not the weights that were cleared.

Those three failures require completely different corrective work, and the trace by itself does not separate them.

The model upgrade problem, generalized

The harness argument does gesture at this. It notes that providers retire models on their own schedule, that replacements do not behave identically, and that the test suite built from old incidents is what tells you whether the new model still respects the confirmation step. Pay special attention to model upgrades.

That instinct is right and the scope is too narrow. A vendor-side model retirement is one instance of a general category: a post-release weight transformation that changes behavior without changing the interface. In an open-weight deployment, most transformations in that category are performed by the deploying organization itself, not announced by a vendor. There is no deprecation email for a quantization that someone ran to make the model fit on the available hardware.

Which means the trigger for re-testing cannot be "the vendor told us." It has to be "the hash changed."

The part the harness argument gets exactly right

One line in the piece deserves to be carried over into evidence generation without modification. Agents are not deterministic, so a scenario that passed once will not necessarily pass again. Run each scenario several times and set a pass threshold that matches the risk.

This is standard practice in agent evaluation and it is underapplied in safety evidence. A finding derived from a single sample of a probabilistic system is a claim about one sample. If that finding is going into a signed artifact that a hospital’s governance committee will rely on, the sampling depth is part of the claim and belongs in the record.

SichGate's findings schema is being extended accordingly: trial count, temperature, seed or an explicit statement that generation was unseeded, and the pass threshold applied per finding. A reader should be able to see not only what the model did but how many times it was asked.

Two layers, two jobs

The honest summary is that these are complementary controls at different layers.

A harness constrains what a failure can reach. Scoped credentials limit the blast radius of a successful prompt injection. Write gates keep an incorrect decision from being applied to a patient record. Escalation paths route the cases the system should not resolve on its own. This is containment, and it is the right response to the fact that no model is perfectly reliable.

Model-integrity evidence establishes what a specific artifact does before it is deployed, and detects when the artifact has changed. This is verification, and it is the right response to the fact that the file on disk is not a constant.

Neither substitutes for the other. A perfect harness around an unverified build contains failures competently without knowing whether the failure rate just tripled. A verified build behind no harness has a documented failure mode and nothing to stop it from reaching a customer record.

The distinction worth holding onto: a harness limits what a failure can reach. It does not tell you the failure is new.