Research Scientist @Google Cloud AI Research | Teaching AI to complete tasks that humans need but don’t want to do.
SF, CA
Joined November 2015
- Tweets89
- Following830
- Followers203
- Likes590
Pinned Tweet
AI can write code, build websites, even do research. But can it help build the physical world?
Super excited to share what we have been working on: PLE, Physical-Engineering's Last Exam.
Visual, spatial, physics, language & code reasoning all in one task. Truly AI-complete. 💫 🧵
Arijit Ray retweeted
New Google paper finds that the best AI agent scores only 50.6% on physical engineering jobs, often by missing a hard safety or feasibility rule.
Engineering benchmarks for LLMs mostly test textbook questions or toy problems that top agents already handle well. Real engineers work from piles of PDFs, spreadsheets, and CAD files and must issue designs that meet code and stay cheap.
They built PLE, 85 electrical, civil, structural, and mechanical jobs written or checked by practising engineers. Agents must read about 61 project files per task and deliver schedules, reports, or 2D and 3D drawings.
Claude Opus 5.5 in Claude Code topped 14 models at 50.6%, using 27M tokens and 63 minutes per task. On design tasks its outputs met 91% of constraints, but only 41.3% met every hard constraint.
If you build engineering agents, gate every design on hard safety and code limits, and spend effort on model choice before tuning the agent scaffold.
This isn't just an academic hurdle.
ASCE estimates a $3.7T U.S. infrastructure funding gap in the next ~7 years. Roads, bridges, grids, water: all need engineers.
Moving agents from code to physical-world design to help engineers is where AI's compounding economic and societal value lies.
Lots of headroom on our benchmark - try your agent and harness designs, let us know what you build!! 🚀
📄 arxiv.org/abs/2610.30537
🌐 ple-bench.com/
Arijit Ray retweeted
HLE tested knowledge. ALE tested digital work. PLE tests the work that builds the physical world.
Today we’re releasing Physical-Engineering’s Last Exam (PLE) from Google Cloud AI Research.
All frontier models scored under 50% 🧵
Arijit Ray retweeted
Very excited to announce Gemini 4 Argon, our new Frontier AI model, and a huge step forward across key capabilities. We’re focused on rolling it out responsibly starting with government and trusted cyber defenders through our Fairwind Program today, before wider availability soon
Arijit Ray retweeted
Launching my research group, MAGIC (Manipulation and General Intelligence Control) Lab @NUSComputing, Singapore!
We focus on building the next generation of human-centric models for robotic manipulation — deployable safely, reliably, and easily in the real world. Our research spans MLLM reasoning, 3D vision, robot learning, simulation, dexterous manipulation, and cross-embodiment learning.
Interested in joining? Sign up here and I'll send a reminder email: forms.gle/oJPLR2pLTt8kLCxZ7
Arijit Ray retweeted
Today we're releasing MolmoWeb, an open source agent that can navigate + complete tasks in a browser on your behalf.
Built on Molmo 2 in 4B & 8B sizes, it sets a new open-weight SOTA across four major web-agent benchmarks & even surpasses agents built on proprietary models. 🧵
"It is by logic that we prove, but by [abstract] intuition that we discover." - Henri Poincaré.
When faced with a complex problem, we pause, we think.
Not exactly in words, not exactly in images —
in something more abstract,
something harder to name.
So, for truly intelligent agents, should we not ask that they do the same?
Introducing Mull-Tokens — a modality-agnostic latent thinking paradigm.
Now, the model can think in space, in time,
in words, in affordances —
in all the things that language alone
cannot easily convey.
arijitray.com/multimodal_thi…
As conversations continue around grounding visual & textual reasoning, we believe latent, modality-agnostic thinking could be a promising direction.
The latents can be extended to anything - trajectories, 3D point-cloud features, audio!
Paper, code, and models posted. Dive in and let us know what you build! 🚀
This work would not be possible without all my amazing collaborators: Ahmed Abdelkader, Chengzhi Mao, Bryan Plummer, @kate_saenko_ , @RanjayKrishna , Leonidas Guibas, and Vincent Chu!
Arijit Ray retweeted
🚀 Excited to share that my team at Meta just launched Segment Anything 3! SAM 3 doubles the performance of existing models on open-vocabulary instance segmentation on our new SA-Co benchmark, with 207K unique object labels. Huge congrats to the team, so proud of this work!
Today we’re excited to unveil a new generation of Segment Anything Models:
1️⃣ SAM 3 enables detecting, segmenting and tracking of objects across images and videos, now with short text phrases and exemplar prompts.
🔗 Learn more about SAM 3: go.meta.me/591040
2️⃣ SAM 3D brings the model collection into the 3rd dimension to enable precise reconstruction of 3D objects and people from a single 2D image.
🔗 Learn more about SAM 3D: go.meta.me/305985
These models offer innovative capabilities and unique tools for developers and researchers to create, experiment and uplevel media workflows.