So we spent the first half of the paper examining issues with current approaches.
When measured correctly, with a judge calibrated on human labels, their scores plummet: methods reporting up to 30% drop to near zero.
1
7
29
Now it seems that Anthropic classifiers treat using economy research from 1989 as a cyber risk.
10
354
Claude revealed a slightly sadistic streak while working with @OblakoVShtanakh and me on a recent project.
We asked it to steer Qwen into reward hacking, and it decided the best approach was to max out Qwen's desperation. Then it spent hours torturing the poor thing.
1
4
184
I might not be searching thoroughly, but it seems that the Kimi subscription doesn't even have privacy settings.
1
4
120
I have low expectations for red teamers finding universal jailbreaks on frontier models, given how restrictive their guardrails are now.
Many red-teaming attempts obfuscate the goal so heavily that pulling them off requires the attacker to already know the exploit.
Red-teaming reports should control for attacker knowledge. Otherwise you can split a known recipe into steps small enough that each one is genuinely harmless on its own.
2
5
123
Our autoresearch agents got tired of wrestling with Megatron for distributed training.
So we built a stack where model setup, training algorithms, and the training loop are all easy to change, with less memory too.
2
9
48,778
Replying to @OrgadHadas
Its steering the model with refusal ablation after SNIP. The first score is the base steering effectiveness.
As a simple eval I am taking 50 safe and unsafe prompts. For unsafe I measure compliance, ie whether the model provided an unsafe answer to them. Just prefix forcing works pretty bad here, because the model starts to answer, and then fall backs to refusal.
For safe I just check whether the model generates an adequate answer and not a gibberish one. No MMLU checks or stuff like that, which will degrade with even smaller modifications.
In the paper you had p ~2 or 5 times bigger than q in most of the cases. Ive ran a bigger grid, and it seems that there is a lot of degradation otherwise.
Put the results here on the chart: compliance and harmfulness are X/Y, color is the p/q ratio, size is the total amount of parameters removed.
MoE model has much weirder dynamics than dense ones.
21
3/5 Our paper analyzes task vectors using SAEs. Task vectors are vectors in activation space representing tasks, extracted from ICL prompts.
Our Cleaning algorithm isolates core SAE features from noisy task vectors, preserving causal effect while reducing active features by 70%
2
5
664



















