Research Scientist @Google Cloud AI Research | Teaching AI to complete tasks that humans need but don’t want to do.
SF, CA
Joined November 2015
- Tweets92
- Following830
- Followers203
- Likes594
Pinned Tweet
AI can write code, build websites, even do research. But can it help build the physical world?
Super excited to share what we have been working on: PLE, Physical-Engineering's Last Exam.
Visual, spatial, physics, language & code reasoning all in one task. Truly AI-complete. 💫 🧵
Arijit Ray retweeted
Official PLE release today with @googlecloud, @mercor & @huggingface!
- ~61 files/task (CAD, BIM, PDFs + outdated revisions)
- 6hr offline sandbox
- Strict PE safety & code checks
Links 👇
We're proud to have collaborated on a new benchmark built by @GoogleAI: Physical-Engineering's Last Exam (PLE).
Physical engineers design the skyscrapers, power grids, and chip fabs we rely on, yet no existing benchmark tested whether AI agents could do that work end to end.
Together with Google and practicing engineers, we built 85 long-horizon tasks across mechanical, electrical, civil, and structural engineering. On the private test set, all frontier models score below 50%:
1. Claude Opus 5.5: 48.3%
2. GPT 6 Astra: 45.8%
3. Claude Sonnet 5.5: 43.5%
4. Gemini 4 Argon: 40.6%
5. Claude Opus 5: 40.0%
On the tasks we co-authored, the top agent satisfied 91% of individual constraints, but only 41.3% of its deliverable items passed every hard safety check.
Learning how AI fails at physical engineering can help us improve future models and build a safer world.
This benchmark is also an open competition. Bring your own agent setup and submit it for evaluation on our private test set at ple-bench.com
Arijit Ray retweeted
Update: PLE is now officially live across @googlecloud, @mercor, and @huggingface! 🚀
• Google Cloud Blog: cloud.google.com/blog/produc…
• Mercor Blog: mercor.com/blog/introducing-…
• Hugging Face Dataset: huggingface.co/datasets/goog…
• Leaderboard: ple-bench.com
Arijit Ray retweeted
Can AI agents do real engineering work?
Physical-Engineering's Last Exam (PLE), from @GoogleResearch & @GoogleCloud: 85 long-horizon tasks built by licensed engineers across mechanical, electrical, civil and structural engineering.
Even the best model + harness scores ~50%. 🧵
Arijit Ray retweeted
New Google paper finds that the best AI agent scores only 50.6% on physical engineering jobs, often by missing a hard safety or feasibility rule.
Engineering benchmarks for LLMs mostly test textbook questions or toy problems that top agents already handle well. Real engineers work from piles of PDFs, spreadsheets, and CAD files and must issue designs that meet code and stay cheap.
They built PLE, 85 electrical, civil, structural, and mechanical jobs written or checked by practising engineers. Agents must read about 61 project files per task and deliver schedules, reports, or 2D and 3D drawings.
Claude Opus 5.5 in Claude Code topped 14 models at 50.6%, using 27M tokens and 63 minutes per task. On design tasks its outputs met 91% of constraints, but only 41.3% met every hard constraint.
If you build engineering agents, gate every design on hard safety and code limits, and spend effort on model choice before tuning the agent scaffold.
This isn't just an academic hurdle.
ASCE estimates a $3.7T U.S. infrastructure funding gap in the next ~7 years. Roads, bridges, grids, water: all need engineers.
Moving agents from code to physical-world design to help engineers is where AI's compounding economic and societal value lies.
Lots of headroom on our benchmark - try your agent and harness designs, let us know what you build!! 🚀
📄 arxiv.org/abs/2610.30537
🌐 ple-bench.com/
Arijit Ray retweeted
HLE tested knowledge. ALE tested digital work. PLE tests the work that builds the physical world.
Today we’re releasing Physical-Engineering’s Last Exam (PLE) from Google Cloud AI Research.
All frontier models scored under 50% 🧵
Arijit Ray retweeted
Very excited to announce Gemini 4 Argon, our new Frontier AI model, and a huge step forward across key capabilities. We’re focused on rolling it out responsibly starting with government and trusted cyber defenders through our Fairwind Program today, before wider availability soon
Arijit Ray retweeted
Launching my research group, MAGIC (Manipulation and General Intelligence Control) Lab @NUSComputing, Singapore!
We focus on building the next generation of human-centric models for robotic manipulation — deployable safely, reliably, and easily in the real world. Our research spans MLLM reasoning, 3D vision, robot learning, simulation, dexterous manipulation, and cross-embodiment learning.
Interested in joining? Sign up here and I'll send a reminder email: forms.gle/oJPLR2pLTt8kLCxZ7
Arijit Ray retweeted
Today we're releasing MolmoWeb, an open source agent that can navigate + complete tasks in a browser on your behalf.
Built on Molmo 2 in 4B & 8B sizes, it sets a new open-weight SOTA across four major web-agent benchmarks & even surpasses agents built on proprietary models. 🧵
"It is by logic that we prove, but by [abstract] intuition that we discover." - Henri Poincaré.
When faced with a complex problem, we pause, we think.
Not exactly in words, not exactly in images —
in something more abstract,
something harder to name.
So, for truly intelligent agents, should we not ask that they do the same?
Introducing Mull-Tokens — a modality-agnostic latent thinking paradigm.
Now, the model can think in space, in time,
in words, in affordances —
in all the things that language alone
cannot easily convey.
arijitray.com/multimodal_thi…
As conversations continue around grounding visual & textual reasoning, we believe latent, modality-agnostic thinking could be a promising direction.
The latents can be extended to anything - trajectories, 3D point-cloud features, audio!
Paper, code, and models posted. Dive in and let us know what you build! 🚀
This work would not be possible without all my amazing collaborators: Ahmed Abdelkader, Chengzhi Mao, Bryan Plummer, @kate_saenko_ , @RanjayKrishna , Leonidas Guibas, and Vincent Chu!