AI Model Distillation Explained: Why Output Harvesting Is Not the Whole Story

Knowledge distillation transfers selected behavior from a larger teacher into a smaller student.

The word distillation is doing too much work in today’s AI debate.

In machine-learning research, it describes a family of methods for transferring knowledge from a teacher model to a student model. In product announcements, it often means training a smaller model from a larger model. In security discussions, the same word may describe collecting millions of responses from a competitor’s API.

Those activities are related, but they are not identical.

I started researching this after watching Prompt Engineering’s video, Distillation Explained: Why It’s So Misunderstood. The video makes an important distinction between access to a model’s probability distribution and access to its final answers. It also connects that distinction to current claims about Chinese AI labs, Claude, and Kimi K3.

After checking the original distillation papers, recent model reports, API documentation, and the public statements behind those claims, my conclusion is:

A finished answer is a much lower-bandwidth teaching signal than a full probability distribution, but it is still a teaching signal. Output harvesting can transfer useful behavior and capability. What it cannot do by itself is reproduce a frontier model’s complete knowledge, architecture, pre-training, and post-training pipeline.

That distinction matters technically, commercially, and politically.

The Short Version

Here is the practical map before I go deeper:

This is a practical classification, not a universal taxonomy. Research papers use overlapping terms, and that terminology is part of the confusion.

What Classic Knowledge Distillation Actually Does

The modern reference point is Geoffrey Hinton, Oriol Vinyals, and Jeff Dean’s 2015 paper, Distilling the Knowledge in a Neural Network.

Imagine an image classifier looking at a handwritten 2.

A normal hard label gives the model one target:

0: 0
1: 0
2: 1
3: 0
4: 0
5: 0
6: 0
7: 0
8: 0
9: 0

That label says what the image is, but it says nothing about resemblance.

A strong teacher model might instead produce a distribution like this:

2: 0.90
7: 0.05
3: 0.02
other classes: 0.03 combined

The wrong answers are informative. If the teacher gives 7 more probability than 3, it may have noticed that this particular 2 has a flat top. The probability distribution contains information about similarities between classes that the one-hot label throws away.

The teacher first produces logits, which are unnormalized scores. Softmax converts them into probabilities:

q_i = exp(z_i / T) / sum_j exp(z_j / T)

T is the temperature.

• At T = 1, the normal distribution may be very sharp.
• At a higher temperature, the distribution becomes softer.
• The softer distribution makes the relative probabilities of non-winning classes easier for the student to learn.

A common distillation objective combines two losses:

student loss
= hard-label loss
+ teacher-distribution loss

The student is optimized against both the ground-truth label and the teacher’s softened distribution. The temperature term exposes more information about the teacher’s relative preferences.

The exact formula varies, but the principle is stable: the student learns both the correct answer and the teacher’s pattern of uncertainty.

This is much richer than showing the student only the winning class.

How This Maps to a Language Model

From left to right, the teaching signal becomes less detailed: internal features, token probabilities, and finally the generated sequence.

An autoregressive language model repeatedly solves a classification problem over its vocabulary.

Given a prompt and all previously generated tokens, it assigns a score to every possible next token:

prompt + previous tokens
          |
          v
teacher logits for the vocabulary
          |
          v
temperature + softmax
          |
          v
probability distribution over the next token

Suppose the partial sentence is:

The capital of France is

The generated answer may reveal only:

 Paris

The full teacher distribution may also reveal how it ranked Lyon, Marseille, punctuation, whitespace variants, and thousands of unrelated tokens. This happens at every token position in a response.

That is why full-distribution distillation is such a dense signal. One generated token can carry information about many alternatives, not only the token that was sampled.

Do APIs Expose That Signal?

The answer is not simply yes or no.

Moving from internal access to top-k probabilities and then generated text progressively reduces the information available to the student.

Some model APIs return only generated text. Some can also return a limited list of likely tokens and their log probabilities. For example, the current OpenAI Responses API reference documents top_logprobs values from 0 to 20.

That is useful, but it is not the same as receiving:

• logits for the entire vocabulary
• hidden states from every layer
• attention values
• model weights
• gradients
• the teacher’s training data

So the accurate statement is:

Typical external APIs expose less information than an internal teacher-training setup, sometimes much less. They do not all expose only one token, and limited top-k log probabilities are still not the full model distribution.

This difference affects what kind of distillation is possible and how efficient it can be.

Sequence-Level Distillation Is Still Distillation

The terminology becomes messy when only final responses are available.

Yoon Kim and Alexander Rush’s 2016 paper, Sequence-Level Knowledge Distillation, trained a student on sequences generated by a teacher. Their best translation student ran ten times faster than its teacher with little performance loss.

Instead of matching the complete word-level probability distribution, the student learns from a teacher-generated dataset:

input prompt
    |
    v
teacher generates a complete answer
    |
    v
prompt-answer pair enters the training set
    |
    v
student trains with ordinary cross-entropy

This is often similar operationally to supervised fine-tuning on synthetic data. The label sequence-level distillation describes where the targets came from; SFT describes how the student may be optimized on them.

The signal is sparse compared with full logit matching. It shows one path through a huge space of possible outputs rather than the entire distribution at every step.

But sparse does not mean useless.

DeepSeek provides a concrete open example. Its DeepSeek-R1 repository says the smaller Qwen- and Llama-based distill models were fine-tuned on 800,000 samples curated with DeepSeek-R1. Those models recorded large gains on math and reasoning benchmarks.

That is capability transfer through generated sequences, not merely a copy of tone or formatting.

The reasonable limitation is narrower:

A response-only student learns the behaviors covered by its prompts and teacher outputs. It does not automatically recover everything the teacher knows or the probability structure behind every answer.

Coverage, data quality, filtering, student capacity, base-model quality, and later reinforcement learning all matter.

Output Harvesting Describes Collection, Not the Training Objective

I find it useful to separate two questions:

1. How was the data obtained?
2. How was the model trained with it?

Output harvesting answers the first question. An operator sends prompts to a model, collects its responses, filters or scores them, removes duplicates, and stores the resulting prompt-response pairs.

Output harvesting can build a useful training dataset without exposing the teacher’s weights, hidden states, or full token distributions.

Those pairs can then be used in several ways:

• supervised fine-tuning
• sequence-level distillation
• training a reward model or judge
• generating tasks for reinforcement learning
• preference optimization
• evaluation and adversarial testing

Calling the collection process a “distillation attack” may be reasonable in a security or product-policy context. Calling the later training sequence distillation may also be reasonable in an academic context. The terms describe different layers of the same pipeline.

This is also why distillation is not automatically malicious.

• A lab distilling its own teacher into a cheaper model is routine engineering.
• A project using outputs under a license that explicitly permits distillation may be legitimate.
• A competitor using fraudulent accounts, bypassing regional restrictions, or violating a provider’s terms creates a separate access, contract, and security issue.

The mathematical operation does not decide whether the data collection was authorized.

Why On-Policy Distillation Is Different

Sequence-level distillation usually starts with teacher-generated answers. On-policy distillation instead asks the student to visit its own states.

Sequence-level distillation trains on fixed teacher outputs. On-policy distillation repeatedly evaluates responses produced by the current student.

A simplified loop looks like this:

1. Student generates a response.
2. Teacher evaluates token choices along that student response.
3. Student updates toward the teacher's distribution or dense feedback.
4. Repeat with the improved student.

This is valuable because the teacher corrects errors where the student actually makes them. The student is not limited to copying ideal teacher trajectories that it may never reproduce during inference.

The Qwen3 technical report provides a useful comparison. Starting from the same off-policy-distilled 8B checkpoint, Qwen reported:

These are Qwen’s reported results for its own setup, not a universal law. Still, they show why labs care about access to dense teacher signals: one teacher evaluation can provide feedback at many token positions, while reinforcement learning may require many sampled rollouts for a much sparser reward.

Distillation Is One Part of a Much Larger Pipeline

A modern frontier model is not usually created in one step.

The high-level pipeline looks more like this:

Pre-training
    broad language, code, knowledge, and multimodal capability
        |
        v
Mid-training or continued pre-training
    long context, domains, data mixtures, or targeted capabilities
        |
        v
Supervised fine-tuning
    instruction following, formats, tool use, and cold-start behavior
        |
        v
Reinforcement learning and distillation
    reasoning, agents, preferences, safety, and specialized skills
        |
        v
Evaluation, filtering, deployment tuning, and iteration

Distillation can appear at several points. It can compress a final model, create synthetic training data, consolidate several specialized teachers, or provide dense feedback during post-training.

But output data alone does not provide:

• a strong base model
• the teacher’s architecture
• its optimizer and training recipe
• its complete pre-training corpus
• its infrastructure
• its internal representations
• its evaluation and data-quality pipeline

A weak base model trained on a narrow collection of excellent answers may become better at those tasks. It does not magically become a copy of the teacher.

The Kimi K3 and Claude Debate

This technical distinction is now part of a geopolitical argument, so it is important to separate documented facts, allegations, and inference.

What is documented

Anthropic published a February 23, 2026 report titled Detecting and preventing distillation attacks. It said DeepSeek, Moonshot AI, and MiniMax collectively generated more than 16 million Claude exchanges through approximately 24,000 fraudulent accounts.

Anthropic attributed more than 3.4 million exchanges to Moonshot and said the traffic targeted agentic reasoning, tool use, coding, data analysis, computer use, and vision. Anthropic says it made the attribution using request metadata and other infrastructure indicators.

That is a serious first-party allegation. The public post does not include the raw requests, account evidence, resulting training sets, or an independent audit, so I would not present every downstream claim as independently proven.

The public product timeline is also documented:

• Anthropic launched Claude Fable 5 on June 9, 2026.
• A US government directive caused Anthropic to suspend access on June 12.
• Anthropic restored global access on July 1 after the controls were lifted.
• Moonshot released Kimi K3 through its products on July 16 and published the full weights and technical report on July 27.

Moonshot’s Kimi K3 technical report describes a 2.8-trillion-parameter mixture-of-experts model with 104 billion activated parameters, native multimodality, and a one-million-token context window. It documents broad pre-training, supervised fine-tuning, domain-specific reinforcement learning, and multi-teacher on-policy distillation that consolidates Moonshot’s specialized policies into one model.

Kimi K3 itself therefore uses distillation. The meaningful question is not whether the word appears in its pipeline. It is which teachers, signals, data, and permissions were involved at each stage.

What the timeline can and cannot tell us

Fable 5 had only short windows of general availability before Kimi K3’s July 16 product release. That makes one claim implausible: Kimi K3’s entire 2.8T pre-training run, architecture, multimodal system, and infrastructure could not have been created from Fable 5 API responses in those few weeks.

But the timeline does not prove that no external outputs influenced any late training stage.

• Anthropic’s Moonshot allegation predates Fable 5 and concerns earlier Claude access.
• A small, targeted post-training dataset can be integrated much faster than a base model can be pre-trained.
• API outputs could be used as answers, tasks, critiques, rubrics, or reward-model data.
• Public benchmarks cannot reveal the provenance of training examples.

The most defensible conclusion is therefore limited:

The public evidence does not support saying that Kimi K3 was simply copied from Fable 5. It also does not let an outside observer rule out the use of Claude-generated data in some part of Moonshot’s broader training pipeline.

Both stronger claims go beyond the public evidence.

Does This Mean Chinese Open Models Will Be Banned?

I could not find a current US rule that imposes a blanket ban on downloading or using all open-weight models from China as of August 13, 2026.

The official America’s AI Action Plan explicitly describes open-weight models as valuable for innovation, private deployment, and research, while also calling for evaluation of frontier models from China for censorship and ideological bias.

That is not the same as saying future restrictions are impossible. Governments can act through:

• chip export controls
• entity restrictions
• federal procurement rules
• access controls on US-hosted frontier models
• cybersecurity reviews
• sanctions or model-specific directives

The temporary Fable 5 directive also shows that model-access policy can move quickly. But concern about distillation, export controls on compute, and a blanket ban on foreign open weights are three different policy questions. They should not be collapsed into one headline.

My Practical Test for Distillation Claims

When I read that one model was “distilled from” another, I now ask six questions:

1. What signal was available? Full logits, top-k log probabilities, hidden features, complete responses, scalar rewards, or only benchmark answers?
2. Who generated the trajectory? The teacher, the student, or an external prompt set?
3. Where in the pipeline was it used? Pre-training, continued training, SFT, RL, reward modeling, or final compression?
4. How broad was the coverage? A narrow coding skill, several million targeted prompts, or a general corpus?
5. What did the student already know? A strong base model can absorb synthetic data much more effectively than a weak model trained from scratch.
6. Was the access authorized? Technical feasibility and contractual permission are separate questions.

Without those details, the word distillation hides more than it explains.

Final Takeaway

The cleanest distinction is not “distillation versus no distillation.” It is the amount and kind of teacher information available.

Full logits provide a dense map of the teacher’s next-token preferences. Hidden states provide even deeper internal signals. Student-generated trajectories let a teacher focus on the student’s mistakes. Finished outputs provide only selected paths—but, at sufficient scale and quality, those paths can still teach reasoning patterns, tool behavior, formatting, domain knowledge, and task-specific capability.

So I would avoid both extremes:

• “A few API answers can clone a frontier model” ignores the enormous contribution of pre-training, architecture, data, infrastructure, and post-training.
• “API outputs can transfer only style” ignores established sequence-distillation research and real open-model results.

The technically honest position sits between them: output harvesting can be valuable and can violate provider rules, but it is not equivalent to receiving the teacher’s weights or full internal distribution. And a release timeline by itself is not a training-data audit.

Sources

• Distillation Explained: Why It’s So Misunderstood — Prompt Engineering
• Distilling the Knowledge in a Neural Network — Hinton, Vinyals, and Dean
• Sequence-Level Knowledge Distillation — Kim and Rush
• Qwen3 Technical Report
• DeepSeek-R1 official repository
• Kimi K3 official repository and technical report
• Detecting and preventing distillation attacks — Anthropic
• Claude Fable 5 and Claude Mythos 5 — Anthropic
• Redeploying Fable 5 — Anthropic
• OpenAI Responses API reference
• America’s AI Action Plan — The White House