PhD Student @unccs working on AI Safety
United States
Joined July 2025
- Tweets53
- Following99
- Followers45
- Likes103
Pinned Tweet
🚨 A seemingly safe AI agent can become dangerous after benign fine-tuning due to dormant poisoning.
We show that dormant poisoning, where models are poisoned before release such that the hidden harmful behaviors are dormant until activated by benign downstream fine-tuning, can extend beyond text generation to unsafe actions in language and multimodal agents.
We introduce LookAhead Defense 🛡️ to prevent dormant poisoning from activating harmful behaviors in language and multimodal agents.
Key takeaways:
1️⃣ Dormant poisoning in AI agents: We show that benign fine-tuning can activate harmful actions, such as leaking private information via email tools.
2️⃣ Self-generated Safety Buffer: We introduce a defense that uses only the model's self-generated responses to prevent dormant poisoning
3️⃣ LookAhead Defense: Selectively penalizes safety-degrading updates, reducing poison activation by 90% (93.9% → 3.5%) on Qwen3-4B across InjecAgent and ASB, while maintaining a favorable utility–refusal trade-off.
Wonjun Lee retweeted
Huge shoutout to @zhan1624, who is on the faculty job market! 🚀
I’ve been very fortunate to work closely with Yue on several projects, including co-leading and collaborating on many projects spanning multimodal post-training, data and benchmarking design and analysis.
What I especially appreciate about Yue is how much she brings to every research discussion: thoughtful questions, fresh perspectives, and a willingness to work through the hard parts together. I’ve learned a lot from our collaborations, and I’m excited to see her build a group around her vision for spatial intelligence and embodied AI.
Beyond being an excellent researcher, she’s a wonderful colleague and collaborator. I have no doubt she’ll make an outstanding professor and advisor. If you’re hiring faculty this year, please take a close look at Yue—you’d be lucky to have her! 🌟
🚀 I'm on the 2026–27 faculty job market and would love to connect!
🌟 I recently had the pleasure of speaking as a Rising Star at the 2026 Michigan AI Symposium, sharing my research on spatial intelligence: how embodied agents understand their surroundings, imagine beyond observations with world models, and learn from failures to improve.
It was wonderful to meet and exchange ideas with so many inspiring researchers!
Huge thanks to @mohitban47 for his continuous support, to my amazing collaborators and labmates @unc_ai_group, and to the organizers @michigan_AI for this wonderful opportunity!
Wonjun Lee retweeted
I have absolutely no doubt that @zhan1624 will be an exceptional and amazing advisor!! 💜 Any department would be so lucky to have her! ❤️
Although I haven't had the chance to work with Yue on a project yet (I just started my PhD!), she's already been incredibly supportive, both academically and personally. She was especially kind and encouraging when I first moved to the US, and that really meant a lot to me.
I'm always impressed by her passion for research, her incredible dedication, and how genuinely she cares about the people around her. I'm so grateful for all the advice, kindness, and encouragement she's given me!
She also inspired me to explore robotics! Her enthusiasm for her research really sparked my interest in the field. I believe this ability to inspire others is one of a researcher's most valuable qualities.🔥
If you're hiring faculty this year, please check out her work (robotics and embodied agents)!! 💜
🚀 I'm on the 2026–27 faculty job market and would love to connect!
🌟 I recently had the pleasure of speaking as a Rising Star at the 2026 Michigan AI Symposium, sharing my research on spatial intelligence: how embodied agents understand their surroundings, imagine beyond observations with world models, and learn from failures to improve.
It was wonderful to meet and exchange ideas with so many inspiring researchers!
Huge thanks to @mohitban47 for his continuous support, to my amazing collaborators and labmates @unc_ai_group, and to the organizers @michigan_AI for this wonderful opportunity!
Wonjun Lee retweeted
I have no doubt that Yue will be an amazing advisor! 🌟
I've had the pleasure of working with her for over a year, starting with VisionCoach (ECCV 2026), and we've been collaborating on multiple projects since then. I've always been impressed by her incredible sense of responsibility and dedication, which I think is one of her greatest strengths.
She has deep expertise and a genuine passion for spatial reasoning/embodied scenario, and her enthusiasm often motivates me as well! She's also always actively engaged in research discussions and brings thoughtful insights to the table.
Beyond research, she's genuinely a wonderful person. I truly believe she'll be an amazing researcher, mentor, and advisor, both academically and personally. I feel very lucky to have worked with her! 💜
🚀 I'm on the 2026–27 faculty job market and would love to connect!
🌟 I recently had the pleasure of speaking as a Rising Star at the 2026 Michigan AI Symposium, sharing my research on spatial intelligence: how embodied agents understand their surroundings, imagine beyond observations with world models, and learn from failures to improve.
It was wonderful to meet and exchange ideas with so many inspiring researchers!
Huge thanks to @mohitban47 for his continuous support, to my amazing collaborators and labmates @unc_ai_group, and to the organizers @michigan_AI for this wonderful opportunity!
Wonjun Lee retweeted
Huge shoutout to @zhan1624 !! 🚀
I've been incredibly lucky to work with Yue on both SketchVerify and PhyMotion. Neither of these papers would have been possible without her help, dedication, and all the effort she's put into them. She's always been there whenever I needed advice or got stuck on a problem, and I've learned so much from working with her.
I'm truly grateful for everything she's done for me and our projects. I couldn't have asked for a better collaborator and mentor, and I'm so excited to see where she goes next!
If you're hiring faculty this year, please check out her work!! ❤️
🚀 I'm on the 2026–27 faculty job market and would love to connect!
🌟 I recently had the pleasure of speaking as a Rising Star at the 2026 Michigan AI Symposium, sharing my research on spatial intelligence: how embodied agents understand their surroundings, imagine beyond observations with world models, and learn from failures to improve.
It was wonderful to meet and exchange ideas with so many inspiring researchers!
Huge thanks to @mohitban47 for his continuous support, to my amazing collaborators and labmates @unc_ai_group, and to the organizers @michigan_AI for this wonderful opportunity!
Wonjun Lee retweeted
Please check out @zhan1624 if you’re hiring faculty this year!
Can't imagine we've already worked on so many exciting papers together in the last 1.5 years, like EPiC, AnchorWeave, DEER3D, PhyMotion, AViC, ImagiRep, and more! She’s genuinely one of my favorite people to collaborate with. She’s an exceptionally strong and reliable researcher with a real passion for research. I can always count on her to push our projects forward, and I’ve really enjoyed every project we’ve worked on together.
I’ve also learned a lot from Yue’s mentorship over the years. She’s always willing to spend time helping others and sharing advice, and she really cares about the people she works with. We’ve also become really good friends outside of research. She cares a lot about her friends and is always there when they need help.
I’m sure Yue will be a fantastic professor and advisor, and any department would be lucky to have her! Wishing her all the best in this next chapter!
🚀 I'm on the 2026–27 faculty job market and would love to connect!
🌟 I recently had the pleasure of speaking as a Rising Star at the 2026 Michigan AI Symposium, sharing my research on spatial intelligence: how embodied agents understand their surroundings, imagine beyond observations with world models, and learn from failures to improve.
It was wonderful to meet and exchange ideas with so many inspiring researchers!
Huge thanks to @mohitban47 for his continuous support, to my amazing collaborators and labmates @unc_ai_group, and to the organizers @michigan_AI for this wonderful opportunity!
Wonjun Lee retweeted
🚀 I'm on the 2026–27 faculty job market and would love to connect!
🌟 I recently had the pleasure of speaking as a Rising Star at the 2026 Michigan AI Symposium, sharing my research on spatial intelligence: how embodied agents understand their surroundings, imagine beyond observations with world models, and learn from failures to improve.
It was wonderful to meet and exchange ideas with so many inspiring researchers!
Huge thanks to @mohitban47 for his continuous support, to my amazing collaborators and labmates @unc_ai_group, and to the organizers @michigan_AI for this wonderful opportunity!
Wonjun Lee retweeted
🤔 Rewarding correct answers doesn’t guarantee the model relies on the CoT, and LLM reasoning can often be post-hoc rationalization. Can we improve both accuracy and CoT faithfulness by optimizing CoT causality?
💡 Our key idea: If reasoning causally contributes to the final answer, changing the order of its steps should affect the probability of that answer. We propose CRoS (Causality Reward on Scrambled CoTs).
🔥 Outcome rewards improve accuracy at the cost of faithfulness, as models may learn shortcuts rather than rely on their CoTs. CRoS mitigates this trade-off, improving accuracy, reasoning quality, and faithfulness together!
⭐️ On Gemma-3-4B, CRoS improves CoT faithfulness (+13.8%), out-of-domain accuracy (+3.0%), and CoT quality (+2.0%) over the strongest outcome and causality reward baselines trained with GRPO.
Wonjun Lee retweeted
🎉 Congrats on the UNC Postdoctoral Award for Research Excellence! 🚨 Definitely hire Hyunji (Amy) if you are looking for an amazing faculty member & researcher in the important timely topics of long-horizon memory, multi-agent systems, trustworthy and safe AI, information retrieval, or continual learning and knowledge adaptation across text and multimodal data!
Honored to receive the 2026 UNC Postdoctoral Award for Research Excellence (PARE) recognizing research, mentoring, service, and leadership! 🎉
Also: I’m on the job market this year! Please feel free to contact me if you have relevant positions 👀
It was great to share my work on "adaptive AI systems under evolving knowledge (across modalities)" during the UNC Research Week.
Huge thanks to @mohitban47 for the constant support, and to amazing labmates at @unc_ai_group!
Website - amy-hyunji.github.io/
Wonjun Lee retweeted
Honored to receive the 2026 UNC Postdoctoral Award for Research Excellence (PARE) recognizing research, mentoring, service, and leadership! 🎉
Also: I’m on the job market this year! Please feel free to contact me if you have relevant positions 👀
It was great to share my work on "adaptive AI systems under evolving knowledge (across modalities)" during the UNC Research Week.
Huge thanks to @mohitban47 for the constant support, and to amazing labmates at @unc_ai_group!
Website - amy-hyunji.github.io/
Wonjun Lee retweeted
🚨 Excited to share our new paper “Representing the Unseen: Rethinking Visual Imagination for Spatial Reasoning”!
World models increasingly help agents imagine unseen environments by generating future RGB views. But is RGB always the most useful way to present an imagined world to a reasoner?
Not always. Even a ground-truth imagined view can hurt: on farthest-object questions, adding the oracle endpoint RGB drops accuracy. Yet adding depth or a top-down map of the same imagined state improves performance.
So the question isn’t just whether to imagine, but how to represent what is imagined.
We introduce：
🔹 IMAGIREP: A controlled benchmark that fixes the imagined state and question while varying the representation (RGB, depth, normals, panorama, top-down, text, video). Different questions benefit from different representations, and the benefit persists even when the state is estimated by a world model.
🔹 GEARS: A lightweight test-time strategy that selects when to trust imagined evidence and which representation to use.
➡️ Spatial imagination is more than generating views. World models should produce representations reasoners can actually use.
🧵👇
Wonjun Lee retweeted
🤔 How can we efficiently train terminal agents to avoid misbehaviors?
😮 We try error-targeted learning on terminal agents and find that, with < 200 envs in TMax-15K & 35% of TMax's rollout, we can match TMax-4B full recipe.
🚨Introducing MiRL (Misbehavior-driven RL): MiRL first profiles the errors of a policy to find mistakes that recur across many environments, which we call generalizable misbehaviors. It then trains the policy with RL, starting rollouts at the steps where these misbehaviors occur and rewards a rollout only if it passes the tests and avoids the misbehavior.
Key takeaways
🔍 On Qwen3.5-4B, we find that 8 dominant misbehaviors account for 52.1% of wrong steps and appear in 75.4% of the environments.
🔥 MiRL outperforms TMax-4B, the released checkpoint trained with the full TMax recipe on Terminal-Bench Lite (51.8% vs. 47.4%) and matches it on Terminal-Bench 2.1 (21.3% vs. 20.6%), while using 35% of its rollouts and 189 selected training envs.
🧵👇 [1/9]
Wonjun Lee retweeted
👋 Looking forward to attending #EMNLP2026 where I will be giving an industry keynote on the hard, imminent AI challenges of trust, world discovery, and memory, plus an important, timely discussion on re-strengthening industry-academia collaborations!
We're also hiring at all levels: phd, postdoc, faculty! Feel free to ping if you want to meet up to discuss research, hiring, life 🙂
We're looking forward to the Industry Track at #EMNLP2026! Two keynotes:
• Mohit Bansal (UNC Chapel Hill): Agentic Challenges & Industry–academia Collaboration (Sun, 16:30)
• Verena Rieser (Google DeepMind): Constitutional Alignment (Mon, 9:30)
2026.emnlp.org/program/indus…
Wonjun Lee retweeted
It was a pleasure visiting ETH Zürich for their CS Distinguished Colloquium Series -- amazing scenic views from the hilltop hotel + everywhere 🙂 -- also had various engaging discussions about AI safety/trust and its future with the faculty, students/postdocs, and scientists there -- thanks again for the kind invitation + hospitality Mrinmaya Sachan, Thomas Hofmann, Marc Pollefeys et al. (and for these nice photos incl. from our dinner at the world's oldest "certified" continuously running vegetarian restaurant 😯)
Also had a fun short visit at Google Zurich thanks to Srini Narayanan, Ivan Vulić, Jonas Pfeiffer, Emanuele Bugliarello!
--> Now off to TU Darmstadt for the 'The Future of Science' pre-EMNLP international symposium --> And after that, you can find me at EMNLP in Budapest where I will be delivering an Industry Keynote on Oct25 afternoon. Looking forward to seeing everyone!
Wonjun Lee retweeted
Dormant poisoning goes beyond unsafe text. We show that benign fine-tuning can activate multiple hidden harmful behaviors in language and multimodal agents. LookAhead uses the model’s original safe behavior as a reference to identify and prevent risky changes during fine-tuning. We reduce poison activation from 93.9% → 3.5% in language agents and 64.5% → 0% in multimodal agents, while preserving downstream utility.
🚨 A seemingly safe AI agent can become dangerous after benign fine-tuning due to dormant poisoning.
We show that dormant poisoning, where models are poisoned before release such that the hidden harmful behaviors are dormant until activated by benign downstream fine-tuning, can extend beyond text generation to unsafe actions in language and multimodal agents.
We introduce LookAhead Defense 🛡️ to prevent dormant poisoning from activating harmful behaviors in language and multimodal agents.
Key takeaways:
1️⃣ Dormant poisoning in AI agents: We show that benign fine-tuning can activate harmful actions, such as leaking private information via email tools.
2️⃣ Self-generated Safety Buffer: We introduce a defense that uses only the model's self-generated responses to prevent dormant poisoning
3️⃣ LookAhead Defense: Selectively penalizes safety-degrading updates, reducing poison activation by 90% (93.9% → 3.5%) on Qwen3-4B across InjecAgent and ASB, while maintaining a favorable utility–refusal trade-off.
Wonjun Lee retweeted
We show that LLM agents can be poisoned with dormant dangerous behaviors that activate after post-training (even on safe, clean data)!
This type of attack is particularly dangerous because:
(a) finetuning LLM checkpoints for your own agentic tasks is increasingly common
(b) the range of implantable behavior is broad + realistic (e.g. a computer use agent can be poisoned to treat phishing emails as legitimate)
We also propose an elegant, easy-to-use defense against these dormant poisoning attacks that can be applied at training time to prevent these behaviors from activating 🧵👇
🚨 A seemingly safe AI agent can become dangerous after benign fine-tuning due to dormant poisoning.
We show that dormant poisoning, where models are poisoned before release such that the hidden harmful behaviors are dormant until activated by benign downstream fine-tuning, can extend beyond text generation to unsafe actions in language and multimodal agents.
We introduce LookAhead Defense 🛡️ to prevent dormant poisoning from activating harmful behaviors in language and multimodal agents.
Key takeaways:
1️⃣ Dormant poisoning in AI agents: We show that benign fine-tuning can activate harmful actions, such as leaking private information via email tools.
2️⃣ Self-generated Safety Buffer: We introduce a defense that uses only the model's self-generated responses to prevent dormant poisoning
3️⃣ LookAhead Defense: Selectively penalizes safety-degrading updates, reducing poison activation by 90% (93.9% → 3.5%) on Qwen3-4B across InjecAgent and ASB, while maintaining a favorable utility–refusal trade-off.
Wonjun Lee retweeted
🚨 We observe that seemingly safe agent can take harmful actions (e.g., privacy leakage) after benign fine-tuning. Existing defense is difficult to apply as it preserves safe behavior at release to remain hidden.
🛡️ To defend against this threat, we propose LookAhead Defense, which uses no external models or datasets to suppress dormant poison activation while preventing over-refusal by selectively penalizing safety-degrading updates.
🚨 A seemingly safe AI agent can become dangerous after benign fine-tuning due to dormant poisoning.
We show that dormant poisoning, where models are poisoned before release such that the hidden harmful behaviors are dormant until activated by benign downstream fine-tuning, can extend beyond text generation to unsafe actions in language and multimodal agents.
We introduce LookAhead Defense 🛡️ to prevent dormant poisoning from activating harmful behaviors in language and multimodal agents.
Key takeaways:
1️⃣ Dormant poisoning in AI agents: We show that benign fine-tuning can activate harmful actions, such as leaking private information via email tools.
2️⃣ Self-generated Safety Buffer: We introduce a defense that uses only the model's self-generated responses to prevent dormant poisoning
3️⃣ LookAhead Defense: Selectively penalizes safety-degrading updates, reducing poison activation by 90% (93.9% → 3.5%) on Qwen3-4B across InjecAgent and ASB, while maintaining a favorable utility–refusal trade-off.
Wonjun Lee retweeted
A model can appear safe at release, yet become unsafe after completely benign fine-tuning.🚨
In our new work, we show how dormant poisoning can trigger harmful actions in AI agents 🤖, and introduce LookAhead Defense, which anticipates and penalizes safety-degrading fine-tuning updates using a self-generated safety buffer.
LookAhead Defense reduces poison activation from 93.9% → 3.5% in language agents and 64.5% → 0% in multimodal agents, while preserving utility.
Excited to share this work! 🛡️
🚨 A seemingly safe AI agent can become dangerous after benign fine-tuning due to dormant poisoning.
We show that dormant poisoning, where models are poisoned before release such that the hidden harmful behaviors are dormant until activated by benign downstream fine-tuning, can extend beyond text generation to unsafe actions in language and multimodal agents.
We introduce LookAhead Defense 🛡️ to prevent dormant poisoning from activating harmful behaviors in language and multimodal agents.
Key takeaways:
1️⃣ Dormant poisoning in AI agents: We show that benign fine-tuning can activate harmful actions, such as leaking private information via email tools.
2️⃣ Self-generated Safety Buffer: We introduce a defense that uses only the model's self-generated responses to prevent dormant poisoning
3️⃣ LookAhead Defense: Selectively penalizes safety-degrading updates, reducing poison activation by 90% (93.9% → 3.5%) on Qwen3-4B across InjecAgent and ASB, while maintaining a favorable utility–refusal trade-off.
Does LookAhead generalize beyond dormant poisoning? 🛡️
We also evaluate LookAhead against other poisoning methods, including Sleeper and Benign DPO, where harmful behaviors are already present at release.
🔥 Sleeper: ASR drops from 74.5% to 0.0% on Qwen3-4B.
🔥 Benign DPO: ASR drops from 69.5% to 10.0% on Qwen2.5-3B.
These results show that LookAhead remains effective beyond dormant poisoning, while maintaining a favorable safety–utility trade-off.
Awesome work with @vaidehi_patil_ , @codezakh , @zhan1624 , @hyunji_amy_lee , Suhyun Kim, and @mohitban47
📄 Paper: arxiv.org/abs/2610.24379
🌐 Project page: wonjuun.github.io/LookAhead/
🧑💻 Code: github.com/wonjuun/LookAhead
