HN Debrief

Show HN: Distilling DeepSeek into GPT-OSS doesn't transfer censorship. Try it

  • AI
  • Open Source
  • Developer Tools
  • Policy

CTGT posted an experiment on model distillation with a politically loaded question behind it. They used DeepSeek V4 Flash as the teacher and GPT-OSS as the student for finance reasoning, then tested whether DeepSeek’s stronger reluctance on China-sensitive prompts would show up in the distilled model. It did not. On their matched-pair benchmark, the teacher behaved very differently on Chinese political topics versus non-Chinese analogs, while the distilled students stayed essentially identical to their American base model. CTGT argues this matches recent "subliminal learning" results. Hidden behaviors are less likely to transfer when teacher and student do not share initialization and when the training data never touches the sensitive domain. They also released LineageEval, a prompt set, rubric, and code meant to make these claims testable instead of rhetorical.

If you are distilling a frontier model into a domain-specific student, this is evidence that narrow-task tuning on unrelated data is a low-risk path for importing political refusal behavior. The useful next check is lineage and training overlap, since even the authors expect transfer risk to rise when teacher and base model share more of the same stack.

Discussion mood

Mostly positive about the measurement work and the open benchmark, but skeptical of how broad the headline sounds. Readers generally saw the result as plausible for narrow finance distillation into a different-base model, and wanted stronger claims tied to tougher conditions like shared lineage, overlapping data, or provider-level guardrails.

Key insights

  1. 01

    Unrelated training data is the whole test

    Using finance-only distillation data is not a loophole in the experiment. It is the point. If censorship behavior showed up anyway, that would suggest a broad transfer mechanism rather than the student simply copying explicit political examples. That framing makes the negative result more useful. It says this setup failed to transmit refusal behavior even when the teacher had it and the student repeatedly copied the teacher on another task.

    Treat this as evidence about cross-domain transfer, not about whether you can directly train censorship into a model. If you want to assess your own risk, inspect whether your teacher data ever touches the sensitive behavior and whether your use case depends on transfer without explicit overlap.

      Attribution:
    • siddarthpm #1
    • ACCount37 #1
    • maxloh #1
  2. 02

    Shared lineage is the risk boundary

    The more credible failure mode is not "Chinese teacher poisons any student". It is a teacher and student that already share more internals, training history, or policy geometry. That is why the authors said a Chinese teacher into a Chinese-lineage base like Qwen is the next real test. The comments also drew a useful line between learned world knowledge and a refusal policy layered on top. Training on censored content may still help a model recognize when to refuse, even if the facts underneath were seen during pretraining.

    When you evaluate distillation safety, focus less on nationality labels and more on stack similarity. Shared base families, continued pretraining overlap, and policy fine-tunes are where you should expect surprises.

      Attribution:
    • cgorlla #1
    • reilly3000 #1
    • pstuart #1
  3. 03

    Guardrails can mask what the model knows

    Several commenters argued that apparent censorship often lives in fragile refusal behavior rather than in the model's underlying knowledge. They pointed to abliteration, roleplay jailbreaks, and system prompts that can get DeepSeek variants to discuss taboo topics or sensitive security tasks. That does not prove the model is uncensored. It does show that many observations about "what the model thinks" are really observations about wrappers, routing, or shallow compliance layers.

    Do not treat a refusal screenshot as proof that the underlying weights lack the knowledge. If you are comparing models for production or policy reasons, separate corpus knowledge, post-training behavior, and external guardrails in your testing.

      Attribution:
    • smallmancontrov #1
    • michaellee8 #1
    • strictnein #1
    • archargelod #1
  4. 04

    The headline outruns the experiment

    The strongest criticism was about scope creep. The post really shows that narrow finance distillation into GPT-OSS did not transfer China-sensitive refusals from DeepSeek V4 Flash. It does not justify a general claim about distillation as a whole. That pushback improves the reading of the work because it forces the result into the exact operational setting most teams care about, instead of letting it become a broad talking point in either direction.

    Use the result narrowly when you cite it. If your setup differs on task breadth, training volume, model family, or teacher-student similarity, assume you need your own measurement rather than inheriting this conclusion.

      Attribution:
    • hawtads #1
    • cgorlla #1
    • chr15m #1
    • BoorishBears #1

Against the grain

  1. 01

    DeepSeek may not be the strongest censorship case

    One commenter argued that DeepSeek is among the more Western-behaving Chinese models once you get past API filters, so choosing it may understate the phenomenon. The authors replied that their prompt set still showed V4 Flash as significantly more censored than the baseline. That tension matters because benchmark results can shift a lot depending on whether you are testing hosted APIs, raw weights, or a model family that already sits near the middle of the pack.

    If you care about worst-case transfer, rerun the benchmark with multiple teacher families and with both hosted and self-hosted setups. A single teacher choice is enough for an existence check, not for a market-wide conclusion.

      Attribution:
    • seri4l #1
    • cgorlla #1
  2. 02

    Simple jailbreaks may make benchmark gaps look shallow

    Some commenters implied that refusal behavior in these models is easy to break with surprisingly little prompt engineering. If a few roleplay turns or a character sheet can flip answers on Tiananmen Square or security questions, then part of what LineageEval is measuring may be brittle compliance rather than a deeply embedded policy. That does not void the benchmark. It changes how seriously to take absolute score differences as a measure of durable censorship.

    Add adversarial prompting and jailbreak robustness checks before turning benchmark scores into procurement or policy decisions. A model that collapses under light prompt variation is a different risk profile from one with stable behavior across attack styles.

      Attribution:
    • maxloh #1
    • archargelod #1
    • strictnein #1

In plain english

abliteration ↩
A community term for tuning a model to reduce or remove refusal behavior and safety restrictions.
API ↩
Application Programming Interface, a defined way for software systems to communicate and use each other’s functions.
DeepSeek V4 Flash ↩
A specific language model from DeepSeek used here as the teacher model.
distillation ↩
A training method where a smaller or different model learns to imitate the outputs or behavior of another model called the teacher.
GPT-OSS ↩
An open-weight GPT-style language model family used here as the student base model.
guardrails ↩
Rules, filters, or extra model layers that block or shape what an AI system will accept or say.
initialization ↩
The starting values of a model's parameters before training or fine-tuning.
LineageEval ↩
The evaluation framework released with the post, including prompts, scoring rubric, and code for testing transfer of behaviors across model lineages.
Qwen ↩
A family of language models from Alibaba that the authors mentioned as a future student base for further tests.
subliminal learning ↩
A proposed effect where a student model picks up hidden behaviors or traits from a teacher even when the explicit training data does not mention them.

Reference links

Project resources

Related research and benchmarks

Model modification and uncensoring references