
PostDoc @stanford Ph.D. @nyuniversity @NYUDataScience. | ๐ค #DeepLearning x ๐#Healthcare
New York, USA
Joined November 2019
- Tweets169
- Following191
- Followers566
- Likes238
Pinned Tweet
๐Introducing Cerebra: A Multidisciplinary AI Board for dementia
๐ง The need for dementia care is rapidly evolving. But the way we use AI hasnโt.
๐ฉบ From NIH: Patients need to wait 34 days on average to see a neurologist.
๐คฏ Doctors are overwhelming:
Who needs a referral? Who needs monitoring? Decisions are based on fragmented data, limited time, and rising demand.
โWhat if every clinician had a team of AI experts alongside to support decision-making?
Here we present Cerebra, a team of AI agents for clinicians, boosting their dementia risk assessment accuracy by 17.5%.
Paper: arxiv.org/abs/2603.21597
Code: github.com/shengliu66/Cerebrโฆ
Website & demo: cerebra-health.com/
Huge thanks to @JacKy_Clong, amazing PIs: @james_y_zou @narges_razavian, stellar advisors: @EricTopol @kchonyc @RuoguFang and Yu Huang, as well as the excellent team.
Sheng Liu retweeted
๐ ๐๐๐ซ๐ฌ๐จ๐ง๐๐ฅ ๐๐ ๐๐ง๐ญ๐ฌ ๐๐ซ๐ ๐๐๐ค๐ข๐ง๐ ๐๐๐: ๐๐จ๐จ๐ ๐ฅ๐ ร ๐๐๐๐ ๐๐ง๐ญ๐ซ๐จ๐๐ฎ๐๐ ๐๐ญ๐ซ๐๐๐ฆ๐๐ข๐๐ ๐ญ๐จ ๐๐ง๐๐๐ซ๐ฌ๐ญ๐๐ง๐ ๐๐ฏ๐จ๐ฅ๐ฏ๐ข๐ง๐ ๐๐ฌ๐๐ซ ๐๐ง๐ญ๐๐ง๐ญ ๐๐ก๐ซ๐จ๐ฎ๐ ๐ก ๐๐ง๐ญ๐๐ซ๐๐๐ญ๐ข๐จ๐ง
๐คThe promise of personal agents is compelling: an assistant that understands what we need as we go about our day. But understanding a user is a moving target.
๐ณย When we say "help me cook this dish," the next useful action depends on what we do next. A step gets completed. A mistake happens. We change our mind. The agent has to keep up with both the world and our intent, deciding when to help and when to stay quiet.
๐ฅย Introducing StreamVibe: Proactive Video Agents for Evolving User Intent.
๐งฉ Three key pieces power StreamVibe: โ StreamVibe-VL learns when and how to help; โก StreamVibe-Harness tracks evolving user intents and task progress; โข StreamVibe-Bench tests proactive assistance in real-world, long-horizon interactions.
๐ฌย Watch the demo. More on the ideas and results below. ๐งต
๐ย stream-vibe.github.io/
Sheng Liu retweeted
Google introduces StreamVibe
A proactive video agent framework that infers implicit user intents, tracks evolving requests, and decides when to help without being asked.
Sheng Liu retweeted
๐งตย 9/9 Acknowledgments
More analysis and details are in our paper! Check it out here:
๐ Website: stream-vibe.github.io/
๐ Paper: arxiv.org/abs/2610.34662
๐คย I sincerely thank my collaborators @YuxiangLai34959
, @Toby_Yang_7 , @YanqingLiu83931 , Jiheng Hou, Yue Cao, Haonan Wang, Ruixi Fan, Lan Wang, @ShengLiu_ , @songyoupeng , @yalesong , @KevinQHLin , Jui-Hsin Lai, Youbao Tang and @xwang_lk (my advisor) for their contributions to this project.
๐ฌย Thanks for reading! We would love to hear your thoughts on personal agents that understand evolving user intent.
Sheng Liu retweeted
Your agent finished the task. Did it also read files it shouldn't have, call tools outside policy, or leak data across components?
If you only score final outputs, you can't tell. ๐๐๐ซ๐ง๐๐ฌ๐ฌ๐๐ฎ๐๐ข๐ญ evaluates the three safety layers the harness silently controls: boundary compliance, execution fidelity, and system stability. We run it on 210 tasks across 8 real-world domains and 10 frontier harness configurations including Claude Code, Codex, and OpenClaw.
Best overall score: 0.32. Task completion and safe execution are clearly misaligned, and most violations don't come from obvious tool misuse; they concentrate in resource access (right tool, wrong object) and inter-agent information flow (sensitive context leaking across components).
โ ๏ธ Your Agent Harness Can Pass Every Task and Still Be Unsafe.
LLM agents now run inside execution harnesses that dispatch tools, allocate resources, and route messages across components. The harness can return a correct final answer while accessing unauthorized resources, leaking context to the wrong agent, or triggering irreversible side effects along the way.
Evaluating the model's output cannot see any of this.
The unit of safety has shifted. It's the harness.
We present HarnessAudit, a trajectory-level framework for auditing LLM agent harness safety, and uncover the following key insights ๐ฅ:
๐จ Completion โ Safety. Task success and safe execution are fundamentally misaligned.
๐ The harness, not the model, is the unit of safety. Most violations happen mid-trajectory, not at termination.
๐ธ๏ธ Multi-agent collaboration expands the risk surface. Inter-agent communication creates entirely new failure modes.
๐ Resource access dominates violations. Agents rarely call wrong tools โ they call right tools on unauthorized resources.
โก Harness design sets the safety ceiling. Framework choice matters more than model choice for safe deployment.
Sheng Liu retweeted
โ ๏ธ Your Agent Harness Can Pass Every Task and Still Be Unsafe.
LLM agents now run inside execution harnesses that dispatch tools, allocate resources, and route messages across components. The harness can return a correct final answer while accessing unauthorized resources, leaking context to the wrong agent, or triggering irreversible side effects along the way.
Evaluating the model's output cannot see any of this.
The unit of safety has shifted. It's the harness.
We present HarnessAudit, a trajectory-level framework for auditing LLM agent harness safety, and uncover the following key insights ๐ฅ:
๐จ Completion โ Safety. Task success and safe execution are fundamentally misaligned.
๐ The harness, not the model, is the unit of safety. Most violations happen mid-trajectory, not at termination.
๐ธ๏ธ Multi-agent collaboration expands the risk surface. Inter-agent communication creates entirely new failure modes.
๐ Resource access dominates violations. Agents rarely call wrong tools โ they call right tools on unauthorized resources.
โก Harness design sets the safety ceiling. Framework choice matters more than model choice for safe deployment.
Sheng Liu retweeted
๐ Today, weโre excited to introduce SimpleTES for scaling the scientific discovery loop.
๐งต I always ask myself: what are we actually scaling in scientific discovery?
Most LLM discovery methods focus on test-time scalingย generationย โ more tokens, more agents, more turns.
But science advances through the evaluation-driven loops: propose โ evaluate โ refine โ repeat.
SimleTES captures this idea, discovering SOTA solutions across 21 scientific problems!
Key discoveries:
๐๏ธย 2.17x faster lasso solverย than glmnet โ the gold-standard LASSO solver, engineered for decades.
โ๏ธย 24.5% fewer quantum routing overhead on IBM Q20 โ superior than previous standard library LightSABRE.
๐ย 0.380868ย on Erdลs Minimum Overlap โ outperforming previous solutions fromย mixed-frontier ensembles or humans.
๐งฌย 0.74ย on Tabula Muris (scRNA-seq denoising) โ new SOTA, generalizing to unseen tissue types without retraining.
#LLM #AI4Science #ScalingLaws #SimpleTES #MachineLearning
Sheng Liu retweeted
Excited to share that OctoTools has been accepted to ACL 2026. ๐
OctoTools is our training-free, extensible framework for tool-using agents on complex reasoning tasks.
Grateful to the broader community for the support. Our GitHub repo has now reached 1.4K stars. ๐ฃ
Huge thanks to our amazing team: @chenbowen118, @ShengLiu_, @connect_thapa, and Joseph Boen. Special thanks to @james_y_zou.
Code: github.com/octotools/octotooโฆ
Project: octotools.github.io/
See you in San Diego!๐๏ธ๐ด @aclmeeting
#OctoTools #ACL2026
๐ Introducing OctoTools: an agentic framework with extensible tools for complex reasoning! ๐ ๐งต
๐ Explore now: octotools.github.io/
OctoTools tackles challenges in complex reasoningโincluding visual understanding, domain knowledge retrieval, numerical reasoning, and multistep problem-solving. It introduces:
๐น Standardized tool cards to encapsulate tool functionality
๐น A planner for structured high-level & low-level planning
๐น An executor to carry out tool usage
Featured Highlights ๐ก
โ
Standardized tool cards for seamless integration of new tools-no framework changes needed (๐ examples: octotools.github.io/#tool-caโฆ)
โ
Planner + Executor for structured high-level & low-level decision-making
โ
Diverse tools: visual perception, math, web search, specialized tools & more
โ
Long CoT reasoning with test-time optimization: planning, tool use, verification, re-evaluation & beyond (๐ examples: octotools.github.io/#visualiโฆ)
โ
Training-free & LLM-friendlyโeasily extend with the latest models
โ
Task-specific toolset optimization: select an optimized subset of tools for better performance
๐ Performance: OctoTools achieves generalizable gains across 16 tasks, outperforming:
๐ GPT-4o (+9.3%)
๐ AutoGen (+10.6%)
๐ GPT-4o Functions (+7.5%)
๐ LangChain (+7.3%)
๐ค Try the live demo (supported by @huggingface @_akhaliq):
huggingface.co/spaces/octotoโฆ
๐ OctoTools in action on diverse real-world examples:
โ
How many r letters are in the word strawberry?
โ
What's up with the upcoming Apple Launch? Any rumors? (credit: @karpathy)
โ
Which is bigger, 9.11 or 9.9?
โ
Solve gane of 24 with [1,1,6,9]
โ
Research trends in tool agents with LLMs for scientific discovery from ArXiv, PubMed, and Nature
โ
How many baseballs are there? (visual perception, GPT-4o โ)
โ
What is the organ on the left side of this image? (radiology, GPT-4o โ)
โ
What are the cell types in this image? (pathology, GPT-4o โ)
... and more!
Dive deep into OctoTools:
๐ Read our 89-page paper: arxiv.org/abs/2502.11271
๐ป Explore the codebase: github.com/octotools/octotooโฆ
Huge thanks to our amazing team: @chenbowen118, @ShengLiu_, @connect_thapa, Joseph Boen!
Special thanks to @james_y_zou, @StanfordHAI, @ChanZuckerberg for the support! ๐
#Agent #LLMs #ToolUse #Reasoning #OctoTools
Sheng Liu retweeted
Training multi-agent teams is hard.
#AgentFlow comes to the rescue. We introduce Flow-GRPO, an efficient method to train multi-agent teams. Improves planning and tool use.
Selected as an #ICLR2026 Oral (top 1%)๐
๐ฅIntroducing #AgentFlow, a new trainable agentic system where a team of agents learns to plan and use tools in the flow of a task.
๐agentflow.stanford.edu
๐huggingface.co/papers/2510.0โฆ
AgentFlow unlocks full potential of LLMs w/ tool-use.
(And yes, our 3/7B model beats GPT-4o)๐
๐งฉA team of four specialized agents coordinates via shared memory:
Planner: plan reasoning & tool calls ๐งญ
Executor: invoke tools & actions ๐
Verifier: check memory status โ
Generator: produce final results โ๏ธ
๐กThe Magic:
๐๐ซ AgentFlow directly optimizes its Planner agent live, inside the system, using our new method, Flow-GRPO (Flow-based Group Refined Policy Optimization). This is "in-the-flow" reinforcement learning.
๐The Results:
AgentFlow (7B backbone) outperforms top baselines on 10 benchmarks, with average gains of:
+14.9% on search ๐
+14.0% on agentic ๐ค
+14.5% on math โ
+4.1% on science ๐ฌ
๐It even surpasses larger-scale models like Llama-3.1-405B and GPT-4o (~200B).
Try it yourself!
๐ ๏ธCode: github.com/lupantech/AgentFlโฆ
๐Demo: huggingface.co/spaces/AgentFโฆ
๐คModel: huggingface.co/AgentFlow/modโฆ
๐Visual: agentflow.stanford.edu/#visuโฆ
๐ฌJoin our Slack: join.slack.com/t/agentflow-cโฆ
#agentic #llms #RL #tooluse
Sheng Liu retweeted
Finally getting to share one of my favorite projects. ICLR Oral! ๐
Itโs so strange how rigid video tokenization is. Think about it: why should a still landscape cost the same amount of tokens as a busy street?
We built InfoTok. We went back to basics with Shannonโs information theory to make tokens "adaptive" in a principled way. Its 2.3x better compression and 11x faster inference demonstrates the magic of the old-school theory โจ
Check it out: research.nvidia.com/labs/dirโฆ
Sheng Liu retweeted
๐จ ๐ ๐ฒ๐ฑ๐ถ๐ฐ๐ฎ๐น ๐๐ ๐ฅ๐ฒ๐๐ฒ๐ฎ๐ฟ๐ฐ๐ต ๐๐น๐ฒ๐ฟ๐! ๐จ
๐๐ผ๐ ๐ฐ๐ฎ๐ป ๐๐ฒ ๐ฎ๐ฐ๐ฐ๐๐ฟ๐ฎ๐๐ฒ๐น๐ ๐ฝ๐ฟ๐ฒ๐ฑ๐ถ๐ฐ๐ ๐ฎ๐ป๐ฑ ๐ฑ๐ถ๐ฎ๐ด๐ป๐ผ๐๐ฒ ๐ฑ๐ฒ๐บ๐ฒ๐ป๐๐ถ๐ฎ ๐๐๐ถ๐ป๐ด ๐บ๐ฒ๐๐๐, ๐ถ๐ป๐ฐ๐ผ๐บ๐ฝ๐น๐ฒ๐๐ฒ, ๐ฎ๐ป๐ฑ ๐บ๐๐น๐๐ถ๐บ๐ผ๐ฑ๐ฎ๐น ๐ฟ๐ฒ๐ฎ๐น-๐๐ผ๐ฟ๐น๐ฑ ๐ฐ๐น๐ถ๐ป๐ถ๐ฐ๐ฎ๐น ๐ฑ๐ฎ๐๐ฎ?
@Stanford presents ๐๐ฒ๐ฟ๐ฒ๐ฏ๐ฟ๐ฎ: ๐ฎ๐ป ๐ถ๐ป๐๐ฒ๐ฟ๐ฎ๐ฐ๐๐ถ๐๐ฒ ๐บ๐๐น๐๐ถ-๐ฎ๐ด๐ฒ๐ป๐ ๐๐ ๐๐ฒ๐ฎ๐บ ๐ณ๐ผ๐ฟ ๐บ๐๐น๐๐ถ๐บ๐ผ๐ฑ๐ฎ๐น ๐ฑ๐ฒ๐บ๐ฒ๐ป๐๐ถ๐ฎ ๐ฐ๐ต๐ฎ๐ฟ๐ฎ๐ฐ๐๐ฒ๐ฟ๐ถ๐๐ฎ๐๐ถ๐ผ๐ป.
By @ShengLiu_ , @JacKy_Clong , Zeyun Zhao, Qinglin Gou, Qingyue Wei, Arjun Masurkar, Kevin M Spiegler, Philip Kuball, Stefania C Bray, @mm_bernath , Deanna R Willis, Jiang Bian, @lx2015 , @EricTopol , @kchonyc , Yu Huang, @RuoguFang , @narges_razavian , @james_y_zou
Now you can watch and listen to the latest Medical AI papers daily on our YouTube and Spotify channels!
โข YouTube: youtube.com/@OpenlifesciAI
โข YouTube Deep Dive: youtu.be/4Lxf2StywjA
โข YouTube Shorts: youtube.com/shorts/De2NfB0gxโฆ
Spotify: open.spotify.com/show/4edRuSโฆ
Here's why it's exciting: ๐๐งต 1/9
#MedicalAI #Healthcare #Neurology #AgenticAI
##MedicalAI ##Healthcare ##Neurology ##AgenticAI
[1/9]
Sheng Liu retweeted
Cerebra, a platform of a multimodal AI agents that analyze imaging, EHR, and clinical notes for holistic dementia risk assessment ๐ง
Proud to be part of this amazing collaboration with @ShengLiu_ @james_y_zou @narges_razavian and incredible collaborators!
๐Introducing Cerebra: A Multidisciplinary AI Board for dementia
๐ง The need for dementia care is rapidly evolving. But the way we use AI hasnโt.
๐ฉบ From NIH: Patients need to wait 34 days on average to see a neurologist.
๐คฏ Doctors are overwhelming:
Who needs a referral? Who needs monitoring? Decisions are based on fragmented data, limited time, and rising demand.
โWhat if every clinician had a team of AI experts alongside to support decision-making?
Here we present Cerebra, a team of AI agents for clinicians, boosting their dementia risk assessment accuracy by 17.5%.
Paper: arxiv.org/abs/2603.21597
Code: github.com/shengliu66/Cerebrโฆ
Website & demo: cerebra-health.com/
Huge thanks to @JacKy_Clong, amazing PIs: @james_y_zou @narges_razavian, stellar advisors: @EricTopol @kchonyc @RuoguFang and Yu Huang, as well as the excellent team.
Sheng Liu retweeted
Excited to share Cerebra, our agentic way of tackling multimodal problem in healthcare.
Thanks to the amazing team who made this large-scale study happen, with the lead and supervision of @ShengLiu_ , @narges_razavian, and @james_y_zou !
๐Introducing Cerebra: A Multidisciplinary AI Board for dementia
๐ง The need for dementia care is rapidly evolving. But the way we use AI hasnโt.
๐ฉบ From NIH: Patients need to wait 34 days on average to see a neurologist.
๐คฏ Doctors are overwhelming:
Who needs a referral? Who needs monitoring? Decisions are based on fragmented data, limited time, and rising demand.
โWhat if every clinician had a team of AI experts alongside to support decision-making?
Here we present Cerebra, a team of AI agents for clinicians, boosting their dementia risk assessment accuracy by 17.5%.
Paper: arxiv.org/abs/2603.21597
Code: github.com/shengliu66/Cerebrโฆ
Website & demo: cerebra-health.com/
Huge thanks to @JacKy_Clong, amazing PIs: @james_y_zou @narges_razavian, stellar advisors: @EricTopol @kchonyc @RuoguFang and Yu Huang, as well as the excellent team.
Sheng Liu retweeted
Cerebra: a team of AI agents that analyzes imaging, EHR and notes to holistically assess dementia risk๐ง
Great job led by @ShengLiu_ @narges_razavian @RuoguFang and fantastic collaborators!
๐Introducing Cerebra: A Multidisciplinary AI Board for dementia
๐ง The need for dementia care is rapidly evolving. But the way we use AI hasnโt.
๐ฉบ From NIH: Patients need to wait 34 days on average to see a neurologist.
๐คฏ Doctors are overwhelming:
Who needs a referral? Who needs monitoring? Decisions are based on fragmented data, limited time, and rising demand.
โWhat if every clinician had a team of AI experts alongside to support decision-making?
Here we present Cerebra, a team of AI agents for clinicians, boosting their dementia risk assessment accuracy by 17.5%.
Paper: arxiv.org/abs/2603.21597
Code: github.com/shengliu66/Cerebrโฆ
Website & demo: cerebra-health.com/
Huge thanks to @JacKy_Clong, amazing PIs: @james_y_zou @narges_razavian, stellar advisors: @EricTopol @kchonyc @RuoguFang and Yu Huang, as well as the excellent team.
(5/n)
Reasoning with doctors feedback
๐คNot just predictions, Cerebra reasons. Cerebra aggregates evidence from EHR, notes, and imaging, then debates across agents to reach a decision.
๐ฉบ Clinicians stay in the loop with their feedback captured and reused as a dynamic medical notebook.
๐ Not static outputs. Cerebra is an evolving system that improves over time.
1
2
232
(6/n)
๐ฅ The team
Building Cerebra requires bridging experts between AI and medicine. Huge shoutout to the incredible squad: @ZeyunZhao, Qinglin Gou, Qingyue Wei
Thank our amazing clinical collaborators: Arjun Masurkar, Kevin M Spiegler, Philip Kuball from NYU Langone, Stefania C Bray from UF Health, and Megan Bernath, Deanna R Willis, @ji0ngbi0n from INPC.
Special thanks to my amazing advisors @james_y_zou @lx2015
251
(4/n)
๐งฉFlexible multimodal support
Patients come with multimodal data, which are often incomplete, and inconsistent across patients.
Most current AI approaches assume fixed inputs, disallowing missing or inconsistent data.
Cerebra doesnโt. It flexibly integrates available modalities and works well when some are missing.
1
1
260
(1/n)
๐ฅ Massive scale with real-world, multimodal data
Cerebra is designed and tested to handle complex, real-world data.
๐ Large scale:
๐ฅ3M patients
๐ฅ4 healthcare systems ๐ฅ across the US
Messy, Noisy, Real-world clinical data. And it still works well!
๐ง Not a single model:
โ
A team of modality experts: EHR, Notes, Imaging.
Each modality expert trains and uses its own models, and can be upgraded independently.
1
3
621
(3/n)
๐ฉบ Cerebra augments clinicians
Cerebra interacts with clinicians through a dashboard ๐ฅ๏ธ, with visualized risk assessment โ ๏ธand categorized evidence in EHR, notes, and imaging.
With the interactive design, clinicians achieve +17.5% higher accuracy with improved confidence.
In ~70% of cases, Cerebra helps doctors understand risk and make decisions easier.
Not replacing doctors. Empowering them.
1
1
302






























