how

Sharing our latest work on offline self-distillation: 📑 Self-Policy Distillation (SPD): arxiv.org/pdf/2605.22675 Self-distillation methods often need some external scoring signal. But these signals are becoming harder to source for the strongest frontier models, which are themselves already the "teacher." Activation steering, on a separate track, has shown that intervening on a model's hidden activations at inference time can change behavior without updating weights. Most work applies this to behavioral control — refusal, sycophancy, honesty — using directions extracted by contrasting activations with and without the target behavior. SPD builds on these two threads for capability transfer. It steers self-generated training data by projecting KV activations into a capability-selective subspace, then fine-tunes on the resulting outputs with plain next-token loss. Concretely, there are two phases. First, on a small calibration set (50 examples is enough), SPD computes a loss only on the tokens that determine task success — assert test cases in code, the final numeric answer in math, the option letter after "Answer:", etc. Backpropagating that loss to the K and V activations, then applying SVD, gives a low-rank activation subspace that captures directions most important for correctness. Second, SPD generates self-training data under that subspace. At each decoding step, the K and V activations are projected into the subspace, biasing samples toward capability-relevant directions. The projection is then removed, and the original model is fine-tuned on the resulting prompt–completion pairs with standard next-token loss. Across code, math, and multiple-choice QA — MBPP, CodeAlpaca, GSM8K, SVAMP, MMLU, and BBH — on Qwen2.5-0.5B/7B/14B, Qwen3-4B, and Llama-3.1-8B, SPD achieves up to 13% improvement over other self-distillation baselines without external signals. Furthermore, the out-of-domain transfer is where projection-based steering shows it is reaching deeper into model capability. A KV subspace extracted from QA gradients alone yields +136% relative improvement on GSM8K and +24% on MBPP. It shows that whatever direction the SVD is picking up isn't multiple-choice-specific (e.g., format). It's something more general in the reasoning and problem-solving space. As models outgrow their graders, SPD can be especially useful when stronger teachers are unavailable or expensive, which is also the case as LLMs land into more specialized verticals.