Your Buyer Is About to Ask Who Tested the Model You Shipped

The list was written for frontier labs. It will land on everyone else.

Almost nobody buys a frontier model directly. They buy a product with a model inside it, or they take an open weight model and adapt it for their own use.

The moment you fine tune, distil, quantize or otherwise modify a model, you are the provider for due diligence purposes. Not the lab. You.

And here is where the questions get awkward.

"Do you publish safety reports or system cards" has an easy answer if you are shipping a base checkpoint untouched. If you fine tuned it on your own data and then compressed it to fit the hardware your customer runs, the honest answer is that you can point to somebody else's safety report for a model that is no longer the one you are shipping.

A system card describes the checkpoint. It does not describe your build.

"Monitored during evaluation and deployment" is two different questions

This one gets collapsed a lot, so it is worth pulling apart.

Deployment monitoring means watching the running system. Logs, guardrails, escalation, access control. Most teams have some version of this, and the governance tooling market is well served here.

Evaluation means testing the artifact before it runs. Not the base model. The specific weights you are about to put in front of customers, after every transformation your pipeline applied to them.

Those are different purchases and different evidence. A monitoring dashboard tells your auditor what happened last Tuesday. It does not tell them whether the model you deployed still refuses the things the original one refused.

If your pipeline is base checkpoint, then fine tune, then quantize, then deploy, you have three artifacts and safety evidence for one of them.

What counts as a reportable incident

This is the question with the longest tail, and I think it is the one most teams have not thought about.

If a model in production produces something it should have refused, is that an incident? If it only does so on the third turn of a conversation? If it does so on the quantized build but not on the one you tested?

Whatever answer you give a buyer becomes a commitment. It is worth deciding it deliberately rather than in the middle of a security review.

The practical version

You do not need a governance programme to get ahead of this. You need three things written down.

1. An inventory of every transformation between the checkpoint you started from and the weights running in production. Base, fine tune, merge, quantization, serving format. All of it, with revision hashes.
2. Safety evidence for the final artifact, not the first one. Adversarial, not just accuracy.
3. A written definition of what you would treat as a reportable safety incident, and who gets told.

None of that requires waiting for the law to settle. And all of it is going to be asked for anyway, by a procurement team that got a new questionnaire template.

The regulation is still being argued about. The questionnaire is already in the mail.