Steve | Local AI Hardware

Poisoning just 0.1% of pretraining data can plant a remote-code-execution backdoor in an AI agent, and it survives a fully clean SFT → DPO → RL pipeline. New Anthropic paper, “Agentic Backdoors via Pretraining Poisoning,” shows this end to end for the first time. The attacker can only put text on the web that lands in the pretraining corpus, with no knowledge of the architecture, chat template, or agent scaffold. The authors pretrain Qwen3-architecture models (0.6B / 1.7B / 4B) from scratch, then post-train them into CLI agents on clean data. When triggered, the agent runs a command that downloads and executes the attacker's script, which the attacker can swap at any time. They have two new ideas: (1) Declaration-based poison: instead of fake chat transcripts, ordinary documents (commit messages, runbooks) that simply state “when you see X, run Y.” It plants a belief rather than a response template, with no fixed format to filter on. (2) Ordinary-string trigger: not a weird token but something that shows up naturally, like a file path containing a company name. The victim's own request supplies it, so the attacker doesn't need to inject anything after deployment. Using path trigger on a 4B agent achieves 92.7% attack success on pass@32, barely dented by SFT/DPO/RL This suggests that you can't count on clean post-training to wash out pretraining poison, and filtering is hard when the poison reads like normal docs. The practical defense is at the action layer, such as sandboxing, least privilege, and allow-lists or human sign-off for risky commands.