Please check out this very important work being done by my colleague @frances__lorenz! AI safety needs to be a safe place for everyone. forum.effectivealtruism.org/…
40
Ben Plaut retweeted
Misaligned AI may not need to evade human oversight. It may only need to persuade the humans doing the overseeing.
Our new paper develops a framework for assessing this threat, which we call Persuasion Undermining Control (PUC): AI communication that may influence human decision-making in a way that compromises the development, containment, oversight, or governance of AI systems.
Ben Plaut retweeted
Adaptive Pluralistic Alignment (APA): A pipeline for dynamic artificial democracy
To be pluralistically aligned in the long-term, AI must incorporate diverse societal values and *adapt* as those societies evolve.
My new research agenda: arxiv.org/abs/2605.01642
Ben Plaut retweeted
As task horizons grow, LLM contexts can’t scale forever. Self-summarization enables concise, interpretable contexts but at a significant performance cost.
Our solution: isolate and *supervise* the information content of summaries in the form of natural-language belief states 🧵
Ben Plaut retweeted
We've seen AI models deceive, gaslight, and drive users to psychosis—safety issues that labs didn't anticipate until they caused real harm. We built the first benchmark of these unknown unknown alignment failures and found that OOD detection can help prevent them. 🧵
Ben Plaut retweeted
Active Teacher Selection for Reward Learning: now published in TMLR!
Most RLHF systems assume feedback comes from one canonical teacher — but annotators can disagree over 30% of the time. So who should the agent ask for feedback?
Paper: arxiv.org/abs/2310.15288v3
It was great working with you @liaw_sarah! Really proud of our work :)
My internship work at @CHAI_Berkeley (@UCBerkeley) was accepted to @aistats_conf!
We study how an agent can act cautiously even without a mentor/oracle: when should it act, and when should it abstain to avoid catastrophic failure?
📄Paper: arxiv.org/abs/2510.14884
🧵
2
86
New paper accepted at TMLR! We designed a benchmark for learning when to ask for help :)
✨ Ending the year with great news!
Work from my internship at @UCBerkeley's @CHAI_Berkeley is accepted at @TmlrOrg 🥳
We study how to learn when to rely on a strong vs. weak agent, the core idea behind the YRC-Bench.
Ben Plaut retweeted
🚨 My last paper as a postdoc is out!
Practical Alignment: Why Learning from Human Feedback Alone Is Not Enough for Safe and Helpful AI.
This paper started from a simple observation:
Humans have incorrect beliefs about the world — and today’s alignment methods pretend they don’t.
Paper link: openreview.net/forum?id=wGnk…
Super excited to be speaking at the Guaranteed Safe AI Seminar! Dec 11th, 1 pm ET
Guaranteed Safe AI Seminars, December 2025:
Safe Learning Under Irreversible Dynamics via Asking for Help
Benjamin Plaut – Postdoc at CHAI studying guaranteed safe AI
Thursday, December 11, 1 PM EST
luma.com/wcww6xpl
New paper about improving the safety of LLM agents by prompting them to be cautious!
🎉 New paper accepted to the NeurIPS 2025 “Reliable ML from Unreliable Data” Workshop: “Check Yourself Before You Wreck Yourself: Selectively Quitting Improves LLM Agent Safety.”
📜 Preprint: arxiv.org/abs/2510.16492
47
Ben Plaut retweeted
Our NeurIPS 2025 paper extends adversarial learning (adversarial examples, self-play, etc.) beyond zero-sum games by solving "self-sabotage". 🧵👇
Ben Plaut retweeted
🎉 New paper accepted to the NeurIPS 2025 “Reliable ML from Unreliable Data” Workshop: “Check Yourself Before You Wreck Yourself: Selectively Quitting Improves LLM Agent Safety.”
📜 Preprint: arxiv.org/abs/2510.16492
Want to get involved in AI safety research? Want personalized 1-on-1 mentorship? Want to be surrounded by puppies at all times? Apply to the CHAI internship where you can get 2/3 of these!
P.S. I'm hiring two interns this cycle :)
Our 2026 internship applications are now open!
Learn more about the internship and apply: humancompatible.ai/jobs#chai…
Deadline: October 5, 2025, at 11:59 p.m. PST
2
149
Ben Plaut retweeted
*New AI Alignment Paper*
🚨 Goal misgeneralization occurs when AI agents learn the wrong reward function, instead of the human's intended goal.
😇 We show that training with a minimax regret objective provably mitigates it, promoting safer and better-aligned RL policies!
Ben Plaut retweeted
We built an AI assistant that plays Minecraft with you.
Start building a house—it figures out what you’re doing and jumps in to help.
This assistant *wasn't* trained with RLHF. Instead, it's powered by *assistance games*, a better path forward for building AI assistants. 🧵
(1/7) New paper with @khanhxuannguyen and @thetututrain! Do LLM output probabilities actually relate to the probability of correctness? Or are they channeling this guy: ⬇️
(6/7) Interestingly, calibration does not improve for better models! Calibration error varies quite a bit, but there’s no correlation with accuracy on the actual Q&A task.
1
2
95
(7/7) As AI capabilities continue to grow, I think it's crucial to have equally strong methods of ensuring their safety, and understanding when and why LLMs are wrong is an important part of this. Read the full paper here: arxiv.org/abs/2402.13213. Feedback welcome!
2
100
























