
Principal ML Scientist & Tech Lead @Genentech @Roche. PhD @UCLA. Building agentic AI platforms for drug discovery at scale
New York City
Joined September 2011
- Tweets195
- Following1K
- Followers462
- Likes974
Mingyu Derek Ma retweeted
๐ขIntroducing BiomniBench โ the first benchmark focused on evaluating the process, not just the final answer, of AI agents on long-horizon biology research tasks.
We evaluate whether each analysis step is statistically sound and scientifically defensible โ just like peer review, where the validity of the process matters, not matching a single ground-truth answer.
Built with first authors from Nature, Cell, and Science papers, BiomniBench captures 100 most ambitious research tasks across disease areas.
Huge headroom for performance, jagged frontiers across tasks, and many more interesting results:
๐๐ฎ๐ป ๐๐ ๐ฎ๐ด๐ฒ๐ป๐๐ ๐ฝ๐ฒ๐ฟ๐ณ๐ผ๐ฟ๐บ ๐ฏ๐ถ๐ผ๐บ๐ฒ๐ฑ๐ถ๐ฐ๐ฎ๐น ๐ฑ๐ฎ๐๐ฎ ๐ฎ๐ป๐ฎ๐น๐๐๐ถ๐ ๐๐ฎ๐๐ธ๐ ๐ฏ๐ฒ๐ต๐ถ๐ป๐ฑ ๐ฝ๐ฎ๐ฝ๐ฒ๐ฟ๐ ๐ถ๐ป ๐ก๐ฎ๐๐๐ฟ๐ฒ, ๐๐ฒ๐น๐น, ๐ฎ๐ป๐ฑ ๐ฆ๐ฐ๐ถ๐ฒ๐ป๐ฐ๐ฒ?
To find out, we built ๐๐ถ๐ผ๐บ๐ป๐ถ๐๐ฒ๐ป๐ฐ๐ต, a benchmark we co-developed with the original paper authors and 5+year domain experts to grade AI agents the way a peer reviewer reads a paper: scrutinizing methods, reasoning, and every analytical choice, not just the final answer.
As the first track of this benchmark, ๐๐ถ๐ผ๐บ๐ป๐ถ๐๐ฒ๐ป๐ฐ๐ต-๐๐ฎ๐๐ฎ๐๐ป๐ฎ๐น๐๐๐ถ๐ contains 100 data-analysis tasks drawn directly from 21 published studies in Nature, Cell, Science, Nature Medicine, and other leading journals. Each task hands the agent a real dataset and a research question, then scores its full analytical trajectory against an expert-authored rubric.
What's inside:
- ๐ญ๐ฌ๐ฌ ๐๐ฎ๐๐ธ๐ ๐ฎ๐ฐ๐ฟ๐ผ๐๐ ๐ฑ ๐ฑ๐ถ๐๐ฒ๐ฎ๐๐ฒ ๐ฎ๐ฟ๐ฒ๐ฎ๐ (๐ผ๐ป๐ฐ๐ผ๐น๐ผ๐ด๐, ๐ถ๐บ๐บ๐๐ป๐ผ๐น๐ผ๐ด๐, ๐ป๐ฒ๐๐ฟ๐ผ๐น๐ผ๐ด๐, ๐บ๐ฒ๐๐ฎ๐ฏ๐ผ๐น๐ถ๐ฐ & ๐ฒ๐ป๐ฑ๐ผ๐ฐ๐ฟ๐ถ๐ป๐ฒ, ๐ฐ๐ฎ๐ฟ๐ฑ๐ถ๐ผ๐๐ฎ๐๐ฐ๐๐น๐ฎ๐ฟ) ๐ฝ๐น๐๐ ๐ด๐ฒ๐ป๐ฒ๐ฟ๐ฎ๐น ๐ฏ๐ถ๐ผ๐น๐ผ๐ด๐
- ๐ญ๐ณ ๐ฎ๐ป๐ฎ๐น๐๐๐ถ๐ฐ๐ฎ๐น ๐๐ฎ๐๐ธ ๐๐๐ฝ๐ฒ๐ (๐ฒ.๐ด., ๐๐ช๐๐ฆ/๐ฒ๐ค๐ง๐ ๐ฐ๐ผ๐น๐ผ๐ฐ๐ฎ๐น๐ถ๐๐ฎ๐๐ถ๐ผ๐ป, ๐ง-๐ฐ๐ฒ๐น๐น ๐ฟ๐ฒ๐ฐ๐ฒ๐ฝ๐๐ผ๐ฟ ๐ฟ๐ฒ๐ฝ๐ฒ๐ฟ๐๐ผ๐ถ๐ฟ๐ฒ ๐ฎ๐ป๐ฎ๐น๐๐๐ถ๐, ๐ฐ๐ฒ๐น๐น-๐ฐ๐ฒ๐น๐น ๐ฐ๐ผ๐บ๐บ๐๐ป๐ถ๐ฐ๐ฎ๐๐ถ๐ผ๐ป)
- ๐๐ป ๐ฒ๐
๐ฝ๐ฒ๐ฟ๐-๐ฐ๐๐ฟ๐ฎ๐๐ฒ๐ฑ ๐ฟ๐๐ฏ๐ฟ๐ถ๐ฐ ๐ณ๐ผ๐ฟ ๐ฒ๐๐ฒ๐ฟ๐ ๐๐ฎ๐๐ธ, ๐๐ฐ๐ผ๐ฟ๐ถ๐ป๐ด ๐ฒ ๐ฑ๐ถ๐บ๐ฒ๐ป๐๐ถ๐ผ๐ป๐ ๐ผ๐ณ ๐ฎ๐ป๐ฎ๐น๐๐๐ถ๐ฐ๐ฎ๐น ๐พ๐๐ฎ๐น๐ถ๐๐
- ๐ฃ๐ฟ๐ผ๐ฐ๐ฒ๐๐-๐น๐ฒ๐๐ฒ๐น ๐ฒ๐๐ฎ๐น๐๐ฎ๐๐ถ๐ผ๐ป ๐ผ๐ณ ๐ต ๐ณ๐ฟ๐ผ๐ป๐๐ถ๐ฒ๐ฟ ๐๐๐ ๐ (๐๐ฃ๐ง-๐ฑ.๐ฑ, ๐๐น๐ฎ๐๐ฑ๐ฒ ๐ข๐ฝ๐๐ ๐ฐ.๐ณ, ๐ฎ๐บ๐ผ๐ป๐ด ๐ผ๐๐ต๐ฒ๐ฟ๐) ๐ฎ๐ฐ๐ฟ๐ผ๐๐ ๐ฐ ๐ฎ๐ด๐ฒ๐ป๐ ๐ต๐ฎ๐ฟ๐ป๐ฒ๐๐๐ฒ๐ (๐๐น๐ฎ๐๐ฑ๐ฒ ๐๐ผ๐ฑ๐ฒ, ๐๐ผ๐ฑ๐ฒ๐
๐๐๐, ๐ง๐ฒ๐ฟ๐บ๐ถ๐ป๐๐-๐ฎ, ๐๐ฒ๐บ๐ถ๐ป๐ถ ๐๐๐)
Headline results:
- ๐๐ฟ๐ผ๐ป๐๐ถ๐ฒ๐ฟ ๐บ๐ผ๐ฑ๐ฒ๐น๐ ๐น๐ฒ๐ฎ๐ฑ ๐ฎ๐ ๐ณ๐ฏ.๐ฏ/๐ญ๐ฌ๐ฌ, ๐๐ถ๐๐ต ๐๐๐ฏ๐๐๐ฎ๐ป๐๐ถ๐ฎ๐น ๐ต๐ฒ๐ฎ๐ฑ๐ฟ๐ผ๐ผ๐บ ๐๐ผ ๐ถ๐บ๐ฝ๐ฟ๐ผ๐๐ฒ.
- ๐ง๐ต๐ฒ ๐ฎ๐ด๐ฒ๐ป๐ ๐ต๐ฎ๐ฟ๐ป๐ฒ๐๐ ๐บ๐ฎ๐๐๐ฒ๐ฟ๐ ๐ฎ๐ ๐บ๐๐ฐ๐ต ๐ฎ๐ ๐๐ต๐ฒ ๐ฏ๐ฎ๐๐ฒ ๐บ๐ผ๐ฑ๐ฒ๐น.
- ๐๐ด๐ฒ๐ป๐๐ ๐ณ๐ฎ๐น๐น ๐๐ต๐ผ๐ฟ๐ ๐ผ๐ป ๐ฏ๐ถ๐ผ๐น๐ผ๐ด๐ถ๐ฐ๐ฎ๐น ๐ถ๐ป๐๐ฒ๐ฟ๐ฝ๐ฟ๐ฒ๐๐ฎ๐๐ถ๐ผ๐ป, ๐บ๐ฒ๐๐ต๐ผ๐ฑ ๐๐ฒ๐น๐ฒ๐ฐ๐๐ถ๐ผ๐ป, ๐ฎ๐ป๐ฑ ๐๐ฐ๐ถ๐ฒ๐ป๐๐ถ๐ณ๐ถ๐ฐ ๐ฟ๐ฒ๐ฎ๐๐ผ๐ป๐ถ๐ป๐ด.
We hope to make ๐๐ถ๐ผ๐บ๐ป๐ถ๐๐ฒ๐ป๐ฐ๐ต the most helpful benchmark for biologists to understand how AI agents handle real-world biomedical tasks: where they can be trusted, and where they fall short. We're actively expanding our evaluation effort, and would love to engage the broader scientific community on what comes next.
๐ biorxiv.org/content/10.64898โฆ
๐ค huggingface.co/datasets/phylโฆ
Thanks to our amazing @phylo_bio team (Minta Lu, @TuXinming , @serena2z , @TianweiShe , @lecong , @jure , @KexinHuang5 ) and our collaborators at @LaudeInstitute , @Stanford , @Harvard , @PKU1898 , @virginia_tech , Humanlaya Data Lab, Xbench: @alexgshaw , JOU-HO SHIH, Bingqing Zhao, Minjie Shen, Haochen Yang, Jielin Yan, Rongchuan Zhang, Xinze Wu, Tingting Li, Xiaobo Hu, Yuan Jiang, Jiayun Dong, Tao Peng.
I'm here at #NeurIPS2025 in #SanDiego all week and eager to connect with folks passionate about LLMs, Autonomous Agents, and the frontier of #AI4Science especially AI for Drug Discovery.
๐ We are also hiring ML Scientists/Engineers, SWEs, UX Researchers, Summer Interns at the Foundation Models team, #PrescientDesign @genentech (@Roche) to help us build:๐งชA cutting-edge Scientific Agent platform; ๐งฌAutomation and reasoning engines for real-world drug discovery pipelines; ๐ง Large Language Models trained specifically for molecular and protein tasks.
If you are interested in applying AI to real scientific challenges, let's grab a coffee or chat. Feel free to DM me!
#NeurIPS #MachineLearning #Genentech #Hiring #DrugDiscovery #GenerativeAI #Intern
Our foundation model team is hiring ML Engineers to build next-gen LLM agent platform. Exciting opportunity to push the frontier of AI in science and join a world-class cross-functional team on machine learning and science!
This Post is from an account that no longer exists. Learn more
Mingyu Derek Ma retweeted
๐จ Weโre hiring at Frontier Research @PrescientDesign @genentech! Work on statistical ML for drug discovery:
- Bayesian inference & decision theory
- Uncertainty quantification
- Multi-objective optimization
DMs open, feel free to reach out directly!
๐ tinyurl.com/4bm5st9v
Check out our new work using AI agents for gene panel design, multimodal annotation and cellโcell interaction analysis with performance comparison with human scientists!
๐Meet SpatialAgent: an AI agent for spatial biology (a key tech to understand cancer and develop therapies)
It rocks from experiment design to data analysis to hypothesis generation. It supports multi-organs, spans species, and can work alone or team up with human scientists๐งโ๐ฌ
4
359
๐ The @AAAI Symposium on Agentic AI for Science is coming up Mar 31โApr 2 near SFO! Join us in person or online to hear from top voices in #agent, #LLM, and #science and present your work.
๐ข Weโre still accepting submissions!
๐ aaaiagenticai.github.io
Speakers include @ProfBuehlerMIT, Michael Mahoney, Hanghang Tong, @hengjinlp, @jure, @james_y_zou, Alvaro Velasquez, Erica Briscoe, @lifu_huang, @Yujun_Yan93, @yanliu_usc, Qi Li, Dawei Zhou, @mingyu_ma and @hcwww_ from MIT, Berkeley, UIUC, Stanford, DARPA, Genentech, UC Davis, Dartmouth, USC, VT & UCLA.
Organizers: @aditkulk333 @hcwww_ @lifu_huang Dawei Zhou, @KexinHuang5 @danaikoutra @qingyun_wu @Yujun_Yan93 @nsfzyzz @tprioleau_ahlab @james_y_zou @jure @WeiWang1973
Mingyu Derek Ma retweeted
๐Excited to share our latest work: OpenVLThinker, an exploration into enhancing vision-language models with R1 reasoning capabilities.
By iterative integration of SFT and RL, we enabled LVLMs to exhibit robust R1 reasoning behavior. As a result, OpenVLThinker achieves a 70.2% accuracy on multimodal reasoning benchmarks like MathVista, matching the performance of 72B-scale models.
๐Blog: yihe-deng.notion.site/openvlโฆ
๐คModel: huggingface.co/ydeng9/OpenVLโฆ
๐ปGitHub: github.com/yihedeng9/OpenVLTโฆ
Mingyu Derek Ma retweeted
๐งฌ Excited to share our review "๐๐ซ๐จ๐ญ๐๐ข๐ง ๐๐๐ซ๐ ๐ ๐๐๐ง๐ ๐ฎ๐๐ ๐ ๐๐จ๐๐๐ฅ๐ฌ: ๐ ๐๐จ๐ฆ๐ฉ๐ซ๐๐ก๐๐ง๐ฌ๐ข๐ฏ๐ ๐๐ฎ๐ซ๐ฏ๐๐ฒ".
The survey covers 100+ papers systematically
๐ Paper: arxiv.org/abs/2502.17504
๐ป GitHub: github.com/Yijia-Xiao/Proteiโฆ
#LLM #Survey #Protein #AI #NLP
Mingyu Derek Ma retweeted
Lab-in-the-loop therapeutic antibody design
At @PrescientDesign @genentech we have spent 3+ years reimagining drug discovery. We built a machine learning system to design and execute experiments. Here's how it works and what we can do ๐งต
1/
AAAI Spring Symposium on Agentic AI for Science is calling for papers. We will have a fun 2.5-day event with speakers and participants from agents, LLM and science communities in San Francisco from Mar 31 to Apr 2.
๐ Paper submission deadline: Feb 1, 2025
๐ Both archival and non-archival, long and short papers are welcome
๐ Details: aaaiagenticai.github.io/
Please also consider submitting to workshops on this topic at ICLR (iclragenticai.github.io/) and TheWebConf (webconf2025.github.io/)! Join us in SF!
Organized by an awesome crew: @aditkulk333 @hcwww_ @lifu_huang Dawei Zhou, @KexinHuang5 @danaikoutra @qingyun_wu @Yujun_Yan93 @tprioleau_ahlab @james_y_zou @jure @WeiWang1973
2
342
Mingyu Derek Ma retweeted
Happy New Year everyone! Jim and I just put up our January 2025 release of Speech and Language Processing! Check it out here: web.stanford.edu/~jurafsky/sโฆ
Mingyu Derek Ma retweeted
Do LLMs know the ๐งโ๐คโ๐ง cultural and โ๏ธ legal safety across our globe?
Our #NeurIPS2024 paper ๐ SafeWorld dives into this. We benchmark across 50 countries and 493 regions/races.
We've also cooked up SafeWorldLM, outshining even gpt-4o by a whopping 20% in human evaluations!๐
๐Are LLMs aware of cultural and legal safety in todayโs geo-diverse world?
๐Introducing SafeWorld, our #NeurIPS2024 paper and benchmark assessing LLMsโ understanding of geo-diverse safety, based on cultural norms and policies across 50 countries and 493 regions/races.
โ๏ธWe also propose a multi-dimensional framework for evaluating contextual appropriateness, accuracy, and comprehensiveness, revealing major gaps in current LLMs.
๐งจTo address this, we train SafeWorldLM using DPO, achieving SOTA performance and a 20% higher global human evaluator rating in helpfulness and harmfulness over competing models, including GPT-4o.
๐Paper: arxiv.org/pdf/2412.06483
๐ป GitHub: github.com/PlusLabNLP/SafeWoโฆ
๐ซถ๐ปThis is a joint leading effort with @Wade_Yin9712. Also many thanks to the amazing team @steeve__huang @kaiwei_chang, and @VioletNPeng for their hard work.
Check out more details and results we conclude from our paper in the thread below. ๐งต
๐จ๐ฆ Excited to attend #NeurIPS & #ML4H this week! Iโll be presenting work on LLMs for health and diagnosis, efficient decoding, multimodal reasoning, and more.
Iโm on the academic & industry job market, eager to chat about my on-going efforts on agents for scientific discovery, LLM post-training, and potential opportunities.
Ping me to connect and chat! ๐
Mingyu Derek Ma retweeted
๐ข Iโm recruiting PhD students @CS_UVA for Fall 2025!
๐ฏ Neurosymbolic AI, probabilistic ML, trustworthiness, AI for science. See my website for more details: zzeng.me/
๐ฌ If you're interested, apply and mention my name in your application: engineering.virginia.edu/depโฆ
Mingyu Derek Ma retweeted
Prediction: within the next year there will be a pretty sharp transition of focus in AI from general user adoption to the ability to accelerate science and engineering.
For the past two years it has been about user base and general adoption across the public. This is very natural because user growth is a critical part any business model. But at this point I would say that there is widespread accessibility of LLMs: for most queries from the average person on earth, many LLMs can answer pretty well.
In the upcoming five years I think the focal point will be the ability for AI to accelerate engineering and scientific research, which is the engine of progress in technology. At the frontier of innovation in any field, by definition there will be many open questions and a lot of headroom for better AI to make a difference. The stakes will be very high because progress compounds and also because AI accelerating AI research itself is a strong positive feedback loop.
The other way of saying this is that there is somewhat limited headroom for improving the average user query, but massive headroom for improving the experience for the 1% of queries that would accelerate technological advancement, as well as on queries that people would want to ask the model but currently donโt because models are not smart enough to answer. AI research tends to improve where there is great headroom, and in scientific innovation there be substantial upside.
Our new preprint simulates the ๐คthinking process of the ๐knowledge graph constructors and utilizes the KG structure as inspiration for reasoning. It populates the sparse KG with hypothetical knowledge connections, concretizes the internal knowledge of LLMs, and constructs counterfactual reasoning chains.
๐ Retrieve knowledge as a high-level hint for faithful reasoning, combine parametric and non-parametric knowledge
๐ Prove the ability of LLM to conduct deductive reasoning using very limited information, provide a new perspective for inference time computation
๐ Especially useful in scientific domains where the prompts could be knowledge and reasoning intensive, but building inclusive databases is expensive or infeasible.
Full paper: arxiv.org/pdf/2410.08475
Collaborated with Jiashu He, Jinxuan Fan, @DanRothNLP, @WeiWang1973 and Alejandro Ribeiro from @Penn @UCLA @UCBerkeley @upennnlp
Mingyu Derek Ma retweeted
Big congrats ๐๐๐พ to UCLA PLUS lab @VioletNPeng and collaborators won 3! !outstanding paper awards at #EMNLP2024 ๐๐๐




























