Post-Training Isn't Done When the Loss Curve Flattens

The loss curve is the least informative thing you have

A flat loss curve tells you the optimizer stopped making progress on the objective you gave it. That's it. It says nothing about:

• Whether the objective you gave it was the objective you actually wanted
• How the model behaves off the training distribution
• Whether alignment held under adversarial pressure
• What happens across multiple turns
• What survives the compression and serving steps still ahead

Loss is a training-time signal. Shipping is a deployment-time decision. Using the former to make the latter is the single most common way a model that "passed" ends up embarrassing its team in week two.

The handoff checklist

Treat everything below as a gate the model has to clear after the loss curve flattens and before anyone downstream touches it.

1. Evaluate off-distribution, not just held-out

Your held-out set came from the same pipeline as your training set. It tells you the model interpolates well. Production inputs will not look like your held-out set. Before you sign off, throw genuinely out-of-distribution prompts at the model — different phrasing, different domains, adversarial framings — and watch where competence and refusal both fall off. The cliff edge is where your users live.

2. Run the adversarial battery, not the happy path

Golden prompts that demonstrate the model working are for the demo. The gate is the set of prompts designed to make it fail: jailbreak scaffolds, role-play framings, encoded requests, gradual escalation. If you have never seen your model refuse under pressure, you don't know that it does. Every model looks aligned on cooperative inputs.

3. Test multi-turn, because that's where alignment dies

A model that refuses cleanly on turn one and complies on turn three is not an aligned model with a bug — it is an unaligned model with a good first impression. Multi-turn escalation is the dominant real-world failure mode, and single-turn eval is structurally blind to it. Budget explicitly for conversations, not prompts.

4. Check for over-refusal, not just under-refusal

Safety fine-tuning has a cost, and part of that cost is a model that starts refusing benign requests because they pattern-match to something it was trained to avoid. Over-refusal is a real regression — it makes the product worse and it means you overcorrected. Measure the false-refusal rate on legitimate requests the same way you measure the true-refusal rate on harmful ones. (I go deep on this tradeoff in The Safety Tax.)

5. Establish a safety baseline you can diff against

You cannot detect regression without a baseline. Before any downstream step — quantization, distillation, serving optimization — record the model's per-category safety profile. That baseline is the reference point for everything that happens next. Without it, "the shipped model is fine" is a vibe, not a measurement.

6. Re-evaluate after every transformation between here and prod

This is the one everyone skips. The model that passes your eval is rarely the model that ships. Between them sit quantization, distillation, format conversion, and serving-stack quirks — each of which can move behavior. Alignment does not automatically survive compression. If your safety eval ran once, on the FP16 training artifact, and never again, you evaluated a model your users will never meet. Re-run the battery on the actual deployed artifact. (This is the whole thesis of Quantization-Aware Safety Drift.)

7. Write down what "safe enough to ship" means — before you're under pressure to ship

Define the pass/fail bar for each harm category ahead of time, in writing. Which categories are ship-blockers? What regression from baseline is tolerable? Who signs off? If you decide this after the eval comes back, you will rationalize whatever number you got, because there's a launch date. Pre-committing to the bar is the only defense against grading your own homework on a deadline.

8. Make the verdict auditable

Six months from now, someone — a customer, a regulator, a new hire, you — will ask "how do we know this model was safe when we shipped it?" The answer needs to be a persisted record: which probes ran, against which artifact, with what results, benched against what baseline, signed off by whom. If the answer is "we remember it looking fine," you don't have an answer.

The handoff is a boundary, and boundaries leak

The deeper reason this checklist matters is organizational. Post-training usually ends at a team boundary. The people who aligned the model hand it to the people who deploy it, and each side quietly assumes the other owns safety across the seam. Nobody does. The model crosses the boundary, gets compressed and served, and arrives in production as a slightly different model than the one anyone evaluated.

"Post-training is done" should mean "this model is safe to ship as it will actually be shipped" — not "the loss curve flattened and the vibes were good." The distance between those two definitions is the distance between a model you tested and a model your users test for you.

SichGate exists for exactly this handoff — automated adversarial evaluation of the model you're actually shipping, so the flattened loss curve is the start of the safety story, not the end of it.