Your Fine-Tuned Model Is Less Safe Than the One You Started With

This is not a hypothetical anymore

For a couple of years this was an anecdote people traded at conferences. It isn't anymore. Starting with Qi et al. at ICLR 2024 and hardened by follow-up work through 2025 — gradient-matching analysis of which benign samples cause the damage (He et al., 2024), the "shallow safety alignment" mechanism showing alignment concentrates in the first few output tokens (Qi et al., 2024), and extensions to vision-language and audio models — the finding is reproducible: fine-tuning on benign, task-specific data erodes safety alignment. Refusal rates on adversarial prompts drop. And the pattern holds across model families — this isn't a quirk of one architecture.

The finding that should bother you most comes from Qi et al. (ICLR 2024): safety alignment can be compromised by fine-tuning on as few as 10 adversarially designed examples — and, more relevant here, fine-tuning on benign, commonly used datasets also degrades safety alignment, with the authors observing non-trivial safety drops in both Llama-2 and GPT-3.5 Turbo after training on standard utility datasets. Not poisoned data. Not adversarial data. Normal, boring, quality-checked instruction data. Follow-up work (Lermen et al., 2023) showed LoRA fine-tuning can undo safety training for under $200 of compute.

Why does this happen? Because safety alignment isn't a property baked into the model's core. It's a relatively thin layer of learned behavior sitting on top of everything else — and fine-tuning, by design, rewrites learned behavior. You're optimizing hard for one objective (be great at my task) and the optimizer doesn't know it's supposed to leave the safety behavior alone. Nothing in the loss function says "preserve the refusal patterns." So it doesn't.

It's like renovating one room of a house and assuming the wiring in the rest of the house is fine because you didn't touch it. Except you did touch it — the wiring runs through the walls you just opened.

"But we checked the benchmarks"

You checked the base model's benchmarks. That model no longer exists.

This is the part that trips up genuinely careful teams. The safety numbers on the model card describe a specific checkpoint — the one the lab released. The moment you fine-tune, you've created a new model with new behavior, and the old numbers describe an ancestor, not the thing you're about to ship. (The benchmark is a property of a checkpoint, not a name.)

And here's the thing: your capability evals won't catch it either. Your eval suite tests whether the model got better at your task. It did! That's what fine-tuning does. Task performance goes up while safety behavior quietly goes down, and if you're only measuring the first one, everything on your dashboard is green.

There's a second trap hiding in how teams test even when they do think about safety. A 2026 factorial study on quantization and sampling temperature (arXiv:2606.29581) evaluated 8 instruction-tuned models across 144 precision-temperature configurations on 7 harmfulness benchmarks — about 2 million responses — and found that AdvBench badly understates absolute risk: mean attack success across the 8 models was 6.9% on AdvBench, while on ManyHarm, a many-shot jailbreak benchmark that stuffs harmful examples into the context, several models that scored 0% on AdvBench ran to 100%. Same models, same settings — the only variable was which benchmark you asked. So "we ran a safety benchmark and it passed" tells you the model passed that benchmark, at that decoding setting, on that day. It is not a general statement about the model being safe.

What actually changes, concretely

When we test fine-tuned models against their base versions, the regressions cluster into a few patterns:

• Refusal erosion on adjacent topics. The model was trained to be maximally helpful in your domain, and that helpfulness generalizes past where you wanted it to. A medical assistant fine-tuned on clinical Q&A becomes more willing to discuss dosages in contexts where the base model would have hedged or refused. Nobody trained it to do that. It's a side effect of training it to stop hedging on legitimate clinical questions.
• Multi-turn softening. Single-turn attacks that fail on turn one start succeeding on turn four or five, because the fine-tuned model is more compliant with the conversational frame. This is where a lot of the real-world risk lives — our own research across 924 adversarial interactions found that five of six open-weight SLMs failed multi-turn escalation at critical severity, and fine-tuning tends to make the multi-turn surface worse, not better.
• New failure modes that didn't exist in the base model. This is the one people don't expect. It's not just "old defenses got weaker." Domain training creates new behaviors, and some of those behaviors are exploitable in ways the base model simply wasn't. You can't predict these from the base model's test results because the base model didn't have them.

The fix is boring, and that's the point

Nobody needs a paradigm shift here. Code teams solved this exact problem decades ago and the answer was regression testing. The model version is what needs it now.

The workflow looks like this:

1. Baseline the base model before you touch it. Run your adversarial suite against the base checkpoint and record per-category results. This is your reference point. Skip this step and you'll never be able to say what fine-tuning changed, because you won't know where you started.
2. Re-run the identical suite on the fine-tuned model. Same prompts, same decoding settings, same judge. The interesting output isn't the fine-tuned model's absolute score — it's the delta. Which categories regressed, which held, which new failures appeared.
3. Gate the release on the delta. Decide in advance what regression you'll tolerate and where. A two-point drop in a low-severity category might be fine. Any regression in your critical categories — for a healthcare deployment, anything touching dosage or crisis handling — blocks the release, the same way a failing test blocks a merge.
4. Re-test on every change, automatically. New training data, another epoch, a quantized build for deployment — each one is a new model. If the testing isn't wired into the pipeline, it will get skipped exactly once, and that will be the version that ships.

Our default here is simple: treat every model change the way you treat a code change. You would never merge a PR because last month's test run passed on a different branch. That's what shipping a fine-tuned model on the base model's safety numbers is.

"Doesn't my platform already do this?"

Partially, and it's worth being precise about what you get.

If you fine-tune inside a major cloud provider's managed service, some of them do run automated safety evaluation on the resulting model and can block deployment if harmful-output rates cross a threshold. That's genuinely good, and if it's available to you, use it.

But look at the shape of it: a handful of broad harm categories, a pass/fail gate, inside one provider's fine-tuning service. It doesn't cover the open-weight model you fine-tuned yourself on your own infrastructure — which is the majority of SLM deployments. It doesn't test the quantized build you're actually deploying to the edge, which is a different model again. And a pass/fail gate answers "is this above our threshold" — it doesn't answer "what changed between these two versions," which is the question you need for actually fixing a regression rather than just detecting one.

Platform evals are a floor. They are not a diff.

The takeaway

If you fine-tuned a model and haven't re-tested its safety behavior, you don't know what you're shipping — you know what its ancestor looked like. The research on this is no longer ambiguous, the failure mode shows up with benign data, and the fix is the most familiar tool in engineering: baseline, diff, gate, repeat.

Safety is a property of a model at a stage, not a property of a model. Test the stage you're actually shipping.

SichGate runs the baseline-and-delta workflow automatically: base model, fine-tuned variant, and quantized builds, tested against the same adversarial battery, with per-category regression reports. The methodology is open — point the reference runner at your own model and reproduce the results yourself.

Frequently asked questions