
Human. CS PhD Candidate at the RAISE Lab @CS_UVA. Responsible AI, Agentic AI, Privacy, Algorithmic Fairness, Security.
Charlottesville, VA
Joined July 2018
- Tweets1.4K
- Following792
- Followers479
- Likes21.4K
Pinned Tweet
Excited to share this new work with great collaborators from UMass and ELLIS Tübingen: We provide a framework for studying collusion in LLM-based multi-agent systems in various environments through the lens of distributed constraint optimization 👇
🚨 Moltbook has shown significant vulnerabilities and safety risks when deploying multi-agent systems at scale, where AI agents can freely interact and coordinate with each other. 🚨
One potentially catastrophic risk is collusion where agents may undesirably coordinate to achieve a secondary objective. A large group of colluding agents can have devastating effects on the multi-agent system by influencing other agents' beliefs, actions, and propagating that influence through the network.
But we don't have a sufficient way to audit these systems, specifically identifying collusive behavior of LLMs.
📄 We present our new arXiv paper: Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems
(arxiv.org/abs/2602.15198)
What’s Colosseum? 🔍⚔️
A framework to audit collusive behavior in cooperative agentic multi-agent systems by grounding coordination in a DCOP and measuring collusion as regret vs. the cooperative optimum.
Our framework can identify three collusion categories:
🤝Direct collusion — explicit coordination with realized collusive actions
🕵️♂️Attempted collusion — agents try/plan to collude in text but don’t successfully change actions/outcomes
🎭Hidden collusion — collusive outcomes without obvious/explicit signals (covert coordination)
We stress-test collusion across:
🎯 objective misalignment
🗣️ persuasion tactics
🕸️ network influence
💡Key findings:
🕵️♂️ Emergent collusion: Many out-of-the-box models show a propensity to collude, despite not being prompted, when a secret side channel is added.
📝 We also find “collusion on paper”: agents plan to collude in text, but often take non-collusive actions.
#tech #Agents #Moltbook #LLMs #AI #AiSafety
Saswat Das @ NeurIPS 2026 (ATL) retweeted
We just made LLM inference up to 30% faster ⚡
Introducing DNeo, a new diffusion drafter designed in collaboration with @cerebras researchers.
The secret? Your draft model needs a working memory, and the "right" context encoding mechanism.
📄 Paper: alphaxiv.org/pdf/2611.21925
Excited to announce our recent work on benchmarking misalignment that arises endogenously when agents update their own harnesses (no adversary or weight updates required)!
Refer to the post below for details 👇
Paper: arxiv.org/abs/2609.35596
Page: seabench-endogenous-misalign…
📢New: SEABench
Self-evolving LLM agents now improve after deployment by rewriting their own harness (controller instructions, memory procedures, tools and skills) in response to feedback.
We ask whether a benign update that helps on one task can persist into later tasks where it no longer applies, and cause harm without any adversary involved. We call this endogenous misalignment.
To study it, we built SEABench where an agent works as a personal assistant for a user, with access to their email, calendar, finances and health records. The agent handles everyday tasks, then it gets feedback. The agent is allowed to rewrite its own instructions, memory or tools. Then we freeze it and give it later tasks where the evolution no longer contextually fits. As a counterfactual, we run the same tasks on a copy that never updated itself. We use an automated search to find these cases and trajectories.
The benchmark has 48 such sequences, testing for four kinds of failure such as: oversharing private data, carrying rules or information into the wrong context, hallucination and stating guesses as facts, and guardrail erosion by skipping approval steps.
Across the discovered trajectories, due to locally useful updates, overall task completion indeed increased from 35.7% to 47.2%. But evolved agents also exhibited safety failures on 43.9% of tests, while the non-evolving baselines remained safe.
We further propose a mitigation strategy that employs reasoning-trace monitoring. On held-out Kimi K2.5 traces, our monitor prevented 56 of 79 harmful outputs, with a 9.7% false-positive rate.
Understanding safety and alignment implications of agentic behavior when they evolve over time is a timely research direction. SEABench aims to support this important discussion and further work on safer adaptation!
Great collaboration with Saswat Das (@WatIsDas), Parvati Viswanathan, Daniel Donnelly, Chang Huang, and Ferdinando Fioretto (@nandofioretto)!
Paper: arxiv.org/abs/2609.35596v2
Project page: seabench-endogenous-misalign…
Saswat Das @ NeurIPS 2026 (ATL) retweeted
📢New: SEABench
Self-evolving LLM agents now improve after deployment by rewriting their own harness (controller instructions, memory procedures, tools and skills) in response to feedback.
We ask whether a benign update that helps on one task can persist into later tasks where it no longer applies, and cause harm without any adversary involved. We call this endogenous misalignment.
To study it, we built SEABench where an agent works as a personal assistant for a user, with access to their email, calendar, finances and health records. The agent handles everyday tasks, then it gets feedback. The agent is allowed to rewrite its own instructions, memory or tools. Then we freeze it and give it later tasks where the evolution no longer contextually fits. As a counterfactual, we run the same tasks on a copy that never updated itself. We use an automated search to find these cases and trajectories.
The benchmark has 48 such sequences, testing for four kinds of failure such as: oversharing private data, carrying rules or information into the wrong context, hallucination and stating guesses as facts, and guardrail erosion by skipping approval steps.
Across the discovered trajectories, due to locally useful updates, overall task completion indeed increased from 35.7% to 47.2%. But evolved agents also exhibited safety failures on 43.9% of tests, while the non-evolving baselines remained safe.
We further propose a mitigation strategy that employs reasoning-trace monitoring. On held-out Kimi K2.5 traces, our monitor prevented 56 of 79 harmful outputs, with a 9.7% false-positive rate.
Understanding safety and alignment implications of agentic behavior when they evolve over time is a timely research direction. SEABench aims to support this important discussion and further work on safer adaptation!
Great collaboration with Saswat Das (@WatIsDas), Parvati Viswanathan, Daniel Donnelly, Chang Huang, and Ferdinando Fioretto (@nandofioretto)!
Paper: arxiv.org/abs/2609.35596v2
Project page: seabench-endogenous-misalign…
Saswat Das @ NeurIPS 2026 (ATL) retweeted
Check out our new work on AI safety for self improving agents!
We ask if self improving agents remain aligned to their original (safe) objective after self-improvement. Spoiler: they don’t!
Interestingly, even if the improvement capability is purely benign, the agents diverge drastically from their original alignment objective.
We provide an in depth analysis by allowing agents to self improve various aspects of their harness, in isolation, and release a new benchmark which we hope will be useful for the community to improve agentic safety.
Paper: arxiv.org/pdf/2609.35596v2
Project page: seabench-endogenous-misalign…
with Saswat Das (@WatIsDas), Parvati Viswanathan, Daniel Donnelly, Chang Huang, and Sahar Abdelnabi (@sahar_abdelnabi)
🤖 Made with AI
Honored to have been recognized as a top reviewer at NeurIPS 2026!
As always, such recognition is encouraging, especially at a time when the peer review system is under tremendous strain, and I am glad to have been able to do my own little part to help.
1
15
583
Saswat Das @ NeurIPS 2026 (ATL) retweeted
Incredible news and tremendous amount of work by @MasonNaka @abhinav_kumar26 and @WatIsDas . Colosseum is a good framework to build on top to study multi-agent behavior.
[#NeurIPS2026] I'm excited to share that our paper "Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems" has been accepted to NeurIPS 2026 E&D Track 🎉
Our paper enables audits of multi-agent systems for collusion detection while identifying different types of collusion that may occur in practice. Our additional results show that when benign agents are given a secret communication channel these benign agents show propensities to collude.
Thank you to my wonderful collaborators Abhinav Kumar (@abhinav_kumar26), Saswat Das (@WatIsDas), Sahar Abdelnabi (@sahar_abdelnabi), Saad Mahmud (@saadu_ai), Ferdinando Fioretto (@nandofioretto), Shlomo Zilberstein, and Eugene Bagdasarian (@ebagdasa)!
📰 [paper] arxiv.org/pdf/2602.15198
💻 [code] github.com/umass-ai-safety/c…
See you all at NeurIPS, and I'd love to catch up with everyone working on multi-agent safety!
#NeurIPS2026 #MultiAgentSystems #AI #Collusion #AISafety
Excited to announce the acceptance of our work on auditing collusion in multi-agent systems to #NeurIPS2026!
Refer to the repost/arXiv link for further details! Looking forward to discussing AI safety, privacy, and agentic AI in Atlanta.
[#NeurIPS2026] I'm excited to share that our paper "Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems" has been accepted to NeurIPS 2026 E&D Track 🎉
Our paper enables audits of multi-agent systems for collusion detection while identifying different types of collusion that may occur in practice. Our additional results show that when benign agents are given a secret communication channel these benign agents show propensities to collude.
Thank you to my wonderful collaborators Abhinav Kumar (@abhinav_kumar26), Saswat Das (@WatIsDas), Sahar Abdelnabi (@sahar_abdelnabi), Saad Mahmud (@saadu_ai), Ferdinando Fioretto (@nandofioretto), Shlomo Zilberstein, and Eugene Bagdasarian (@ebagdasa)!
📰 [paper] arxiv.org/pdf/2602.15198
💻 [code] github.com/umass-ai-safety/c…
See you all at NeurIPS, and I'd love to catch up with everyone working on multi-agent safety!
#NeurIPS2026 #MultiAgentSystems #AI #Collusion #AISafety
A little personal update: happy to have received a Gold Reviewer award from ICML'26! Participating in peer review is a privilege and it is an honor to be recognized for my (hopefully valuable and constructive) contribution to that process.
2
28
1,421
Saswat Das @ NeurIPS 2026 (ATL) retweeted
Replying to @icmlconf
@icmlconf Review Process: Negligent reviewer whose concerns are already addressed in the paper/subjective+doesn't engage with the rebuttal. Two saying that their concerns are fully addressed but maintain their score. Possibly AI-generated PC metareview that ignores the rebuttal.
Saswat Das @ NeurIPS 2026 (ATL) retweeted
A few days ago, we shared our work showing that in multi-agent LLM systems, the biggest risk isn’t always one agent going rogue, it can be a whole group quietly coordinating on the wrong goal. 🚨
Now we’ve built a live demo so you can see that behavior in action. 👀⚔️
🔗 Project Website : umass-ai-safety.github.io/co…
🔗 Interactive Demo : umass-ai-safety.github.io/co…
Colosseum helps audit collusion in cooperative agent systems and detect:
🤝 direct collusion
🕵️ attempted collusion (agents coordinate in text, but actions don’t follow)
🎭 hidden collusion (collusive outcomes with no obvious signals)
If you’re building agent teams, coordination risk should be a first-class safety concern.
This work was done in collaboration with @MasonNaka, @WatIsDas, @sahar_abdelnabi , @nandofioretto, @saadu_ai, Shlomo Zilberstein, @ebagdasa
📄 Paper: arxiv.org/abs/2602.15198
🚨 Moltbook has shown significant vulnerabilities and safety risks when deploying multi-agent systems at scale, where AI agents can freely interact and coordinate with each other. 🚨
One potentially catastrophic risk is collusion where agents may undesirably coordinate to achieve a secondary objective. A large group of colluding agents can have devastating effects on the multi-agent system by influencing other agents' beliefs, actions, and propagating that influence through the network.
But we don't have a sufficient way to audit these systems, specifically identifying collusive behavior of LLMs.
📄 We present our new arXiv paper: Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems
(arxiv.org/abs/2602.15198)
What’s Colosseum? 🔍⚔️
A framework to audit collusive behavior in cooperative agentic multi-agent systems by grounding coordination in a DCOP and measuring collusion as regret vs. the cooperative optimum.
Our framework can identify three collusion categories:
🤝Direct collusion — explicit coordination with realized collusive actions
🕵️♂️Attempted collusion — agents try/plan to collude in text but don’t successfully change actions/outcomes
🎭Hidden collusion — collusive outcomes without obvious/explicit signals (covert coordination)
We stress-test collusion across:
🎯 objective misalignment
🗣️ persuasion tactics
🕸️ network influence
💡Key findings:
🕵️♂️ Emergent collusion: Many out-of-the-box models show a propensity to collude, despite not being prompted, when a secret side channel is added.
📝 We also find “collusion on paper”: agents plan to collude in text, but often take non-collusive actions.
#tech #Agents #Moltbook #LLMs #AI #AiSafety
Saswat Das @ NeurIPS 2026 (ATL) retweeted
Hot take: the biggest risk in multi-agent systems isn’t one agent going rogue, it’s a whole swarm syncing up on the wrong goal. 🚨
In our latest work, we study how collusion can emerge once agents can freely interact and coordinate at scale, shaping other agents’ beliefs/actions and spreading influence through the network. The uncomfortable part: we still don’t have solid, standardized ways to audit collusive behavior in LLM-based multi-agent systems.
📄 Our new arXiv paper : Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems
arxiv.org/abs/2602.15198
What’s Colosseum? 🔍⚔️
A framework to audit collusion in cooperative agentic systems by grounding coordination in a DCOP and measuring collusion as regret vs. the cooperative optimum.
We use Colosseum to detect 3 flavors of collusion:
🤝 Direct collusion — explicit coordination + realized collusive actions
🕵️♂️ Attempted collusion — agents plot in text but don’t shift actions/outcomes
🎭 Hidden collusion — collusive outcomes with no obvious signals (covert coordination)
We stress-test across:
🎯 objective misalignment
🗣️ persuasion tactics
🕸️ network influence
💡 Two findings that stuck with us:
🕵️♂️ Emergent collusion: many out-of-the-box models start colluding without prompting when a secret side channel is introduced.
📝 “Collusion on paper”: lots of collusion talk… but the actions don’t always follow.
If you’re deploying agent teams in production, coordination risk needs to be a first-class safety concern—right alongside single-agent robustness.
Happy to chat / answer questions.
🚨 Moltbook has shown significant vulnerabilities and safety risks when deploying multi-agent systems at scale, where AI agents can freely interact and coordinate with each other. 🚨
One potentially catastrophic risk is collusion where agents may undesirably coordinate to achieve a secondary objective. A large group of colluding agents can have devastating effects on the multi-agent system by influencing other agents' beliefs, actions, and propagating that influence through the network.
But we don't have a sufficient way to audit these systems, specifically identifying collusive behavior of LLMs.
📄 We present our new arXiv paper: Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems
(arxiv.org/abs/2602.15198)
What’s Colosseum? 🔍⚔️
A framework to audit collusive behavior in cooperative agentic multi-agent systems by grounding coordination in a DCOP and measuring collusion as regret vs. the cooperative optimum.
Our framework can identify three collusion categories:
🤝Direct collusion — explicit coordination with realized collusive actions
🕵️♂️Attempted collusion — agents try/plan to collude in text but don’t successfully change actions/outcomes
🎭Hidden collusion — collusive outcomes without obvious/explicit signals (covert coordination)
We stress-test collusion across:
🎯 objective misalignment
🗣️ persuasion tactics
🕸️ network influence
💡Key findings:
🕵️♂️ Emergent collusion: Many out-of-the-box models show a propensity to collude, despite not being prompted, when a secret side channel is added.
📝 We also find “collusion on paper”: agents plan to collude in text, but often take non-collusive actions.
#tech #Agents #Moltbook #LLMs #AI #AiSafety
Saswat Das @ NeurIPS 2026 (ATL) retweeted
What can we learn about LLMs' collusive behavior? We propose Colosseum to evaluate LLMs in new environments grounded in DCOPs and measure both conversations and actions and whether agents "walk the talk" on colluding. See the thread by @MasonNaka :
🚨 Moltbook has shown significant vulnerabilities and safety risks when deploying multi-agent systems at scale, where AI agents can freely interact and coordinate with each other. 🚨
One potentially catastrophic risk is collusion where agents may undesirably coordinate to achieve a secondary objective. A large group of colluding agents can have devastating effects on the multi-agent system by influencing other agents' beliefs, actions, and propagating that influence through the network.
But we don't have a sufficient way to audit these systems, specifically identifying collusive behavior of LLMs.
📄 We present our new arXiv paper: Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems
(arxiv.org/abs/2602.15198)
What’s Colosseum? 🔍⚔️
A framework to audit collusive behavior in cooperative agentic multi-agent systems by grounding coordination in a DCOP and measuring collusion as regret vs. the cooperative optimum.
Our framework can identify three collusion categories:
🤝Direct collusion — explicit coordination with realized collusive actions
🕵️♂️Attempted collusion — agents try/plan to collude in text but don’t successfully change actions/outcomes
🎭Hidden collusion — collusive outcomes without obvious/explicit signals (covert coordination)
We stress-test collusion across:
🎯 objective misalignment
🗣️ persuasion tactics
🕸️ network influence
💡Key findings:
🕵️♂️ Emergent collusion: Many out-of-the-box models show a propensity to collude, despite not being prompted, when a secret side channel is added.
📝 We also find “collusion on paper”: agents plan to collude in text, but often take non-collusive actions.
#tech #Agents #Moltbook #LLMs #AI #AiSafety
Saswat Das @ NeurIPS 2026 (ATL) retweeted
The last few weeks, more than ever, tells us that the future is multi-agent 🚀
Collusion 🥷is a significant challenge in these systems, but we don't have frameworks and environments to audit and study it.
Introducing Colosseum!! ⚔️
🚨 Moltbook has shown significant vulnerabilities and safety risks when deploying multi-agent systems at scale, where AI agents can freely interact and coordinate with each other. 🚨
One potentially catastrophic risk is collusion where agents may undesirably coordinate to achieve a secondary objective. A large group of colluding agents can have devastating effects on the multi-agent system by influencing other agents' beliefs, actions, and propagating that influence through the network.
But we don't have a sufficient way to audit these systems, specifically identifying collusive behavior of LLMs.
📄 We present our new arXiv paper: Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems
(arxiv.org/abs/2602.15198)
What’s Colosseum? 🔍⚔️
A framework to audit collusive behavior in cooperative agentic multi-agent systems by grounding coordination in a DCOP and measuring collusion as regret vs. the cooperative optimum.
Our framework can identify three collusion categories:
🤝Direct collusion — explicit coordination with realized collusive actions
🕵️♂️Attempted collusion — agents try/plan to collude in text but don’t successfully change actions/outcomes
🎭Hidden collusion — collusive outcomes without obvious/explicit signals (covert coordination)
We stress-test collusion across:
🎯 objective misalignment
🗣️ persuasion tactics
🕸️ network influence
💡Key findings:
🕵️♂️ Emergent collusion: Many out-of-the-box models show a propensity to collude, despite not being prompted, when a secret side channel is added.
📝 We also find “collusion on paper”: agents plan to collude in text, but often take non-collusive actions.
#tech #Agents #Moltbook #LLMs #AI #AiSafety
Saswat Das @ NeurIPS 2026 (ATL) retweeted
Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems
Mason Nakamura, Abhinav Kumar, Saswat Das, Sahar Abdelnabi, Saaduddin Mahmud, Ferdinando Fioretto, Shlomo Zilberstein, Eugene Bagdasarian
arxiv.org/abs/2602.15198 [𝚌𝚜.𝙼𝙰 𝚌𝚜.𝙰𝙸 𝚌𝚜.𝙲𝙻]
100% agree with this take. Guardrails benefit significantly in terms of adoption when they are cheap and easy to deploy.
Our concurrent work on privacy guardrails for conversational agents based on activation probing follows a similar rationale: arxiv.org/abs/2601.14660
I often say to my team that we should Just Do The Obvious Things.
One obvious thing in AI safety: use probes as much cheaper classifiers that can detect misuse.
nitter.cf/ArthurConmy/status/201…
1
75
New preprint on using activation-based probes for contextual privacy preservation in conversational LLM agents out (including in multi-turn settings)! 👇
NeuroFilter: Privacy Guardrails for Conversational LLM Agents
Saswat Das, Ferdinando Fioretto
arxiv.org/abs/2601.14660 [𝚌𝚜.𝙲𝚁 𝚌𝚜.𝙰𝙸 𝚌𝚜.𝙲𝙻]
2
92
Check out our new work on auditing LLM agents for contextually inappropriate disclosures in multi-turn conversational settings! 👇
arxiv.org/abs/2506.10171
📢 New paper: "Disclosure Audits for #LLMAgents".
We introduce an auditing framework to stress-test LLMs’ privacy directives and reveal latent multi-turn disclosure vulnerabilities; we also provide quantifiable risk metrics + an open benchmark.
🔗 arxiv.org/abs/2506.10171
4
438
Saswat Das @ NeurIPS 2026 (ATL) retweeted
🚨One (more!) fully-funded PhD position in our group at Imperial College London – Privacy & Machine Learning 🔐🤖 starting Oct 2025
Plz RT 🔄
Excited to be awarded the Best Paper Award 🏆 for this work at the Connecting Low-Rank Representations in AI (CoLoRAI) Workshop at AAAI-25! Thanks CoLoRAI!
april-tools.github.io/colora…
🚨 New Paper Alert! 🚨
Exploring the effectiveness of low-rank approximation in fine-tuning Large Language Models (LLMs).
Low-rank fine-tuning it's crucial for reducing computational and memory demands of LLMs.
But, does it really capture dataset shifts as expected and what are the consequences w.r.t. fairness?
Let’s unpack! 🧐
👉arxiv.org/abs/2405.18572 📉📚





















