Breakthrough AI to solve the world's biggest problems. › Join us: https://nitter.gq/t.co/MjUpZpKi8b › Newsletter: https://nitter.gq/t.co/k9gGznrVGL
Seattle, WA
Joined September 2015
- Tweets4.1K
- Following451
- Followers88K
- Likes3.7K
Pinned Tweet
We’re introducing ToyBlocks, an open simulated tabletop for controlled experiments in robot learning.
It enables robotics researchers to experimentally control training data & models, run targeted evals, & trace failures to their causes. 🧵
github.com/allenai/ToyBlocks
Asking how to thicken frosting sliding off warm cupcakes? 🧁 Let them cool first!
That’s the XY problem: seeking a fix that misses the real issue. XYBench tests whether LLMs spot & address it.
🏆 It won the Best Resource Award at EMNLP 2026! 🧵
github.com/allenai/XYBench
To test whether LLMs were struggling to figure out what users wanted, we spelled out each user’s goal alongside their Q.
Even then, responses from all five models steered users away from mistaken approaches less often than human reference answers did.
1
1
622
We built XYBench to test whether prompting or training changes lead to more useful advice from LLMs.
We hope it helps researchers build assistants that guide users toward helpful solutions, even when they don’t know the right questions to ask.
📝 Blog: allenai.org/blog/xybench
NVIDIA used our MolmoSpaces benchmark to evaluate GR00T 2, alongside RoboCasa365, RoboLab, & RoboDojo. Thanks @NVIDIARobotics!
MolmoSpaces is open, so any team can test their robot policies on the same simulated homes, objects, & tasks:
molmospaces.allen.ai/leaderb…
A finer touch for humanoid robots. 🤖
At #CoRL2026, NVIDIA is introducing GR00T 2, an open world action model for humanoids, alongside new open industrial benchmarks from @RLWRLD_ai and @UCBerkeley built on Isaac Lab-Arena.
Explore how NVIDIA’s open world action models, scalable simulation tools and agentic workflows help researchers train, test and refine robot skills, from connecting cables to assembling small components.
Discover more ➡️ nvda.ws/4ixvdZV
Ai2 retweeted
Ai2 / AI for Science is hiring Summer 2027 research interns! Topics:
- Tools for ideation and hypotheses development
- Long-horizon human-AI interaction
- Generative user interfaces
We encourage publishing and public deployments. Apply below by Dec10 for first review cycle 👇
Ai2 retweeted
World models are becoming simulators for evaluating and training robot policies. But how do we compare them?
They output latents, video or simulator states, and image quality says little about task success. In ToyBlocks we compare what matters: interactions and outcomes.
Introducing “Task-Relevant Fidelity in Robotic World Models,” our work on ToyBlocks, to be presented at the LEAP and TAG/G2A workshops at #CoRL2026.🧵
We’re introducing ToyBlocks, an open simulated tabletop for controlled experiments in robot learning.
It enables robotics researchers to experimentally control training data & models, run targeted evals, & trace failures to their causes. 🧵
github.com/allenai/ToyBlocks
Ai2 retweeted
We're hiring interns! :)
Please apply by November 15 (this Sunday) to be considered in the first round of reviews.
You can find more details about our research focus areas and how to apply on the application page!
& we’re accepting intern applications in:
• AI for Science
job-boards.greenhouse.io/the…
• Open Ecosystem
job-boards.greenhouse.io/the…
• Robotics
job-boards.greenhouse.io/the…
We’re hiring across Ai2 👋
We’re growing our AI for science work:
• Research Scientist, Asta
job-boards.greenhouse.io/the…
• Young Investigator, Open Language Models for Biology
job-boards.greenhouse.io/the… 🧵
& we’re accepting intern applications in:
• AI for Science
job-boards.greenhouse.io/the…
• Open Ecosystem
job-boards.greenhouse.io/the…
• Robotics
job-boards.greenhouse.io/the…
Explore the roles & apply—or share with someone who might be a good fit.
→ See all openings: allenai.org/careers
6
14,144
Ai2 retweeted
What could we learn about robot foundation models if we could systematically generate data at scale, with precise control over what varies?
Excited to see ToyBlocks launch as a principled step toward answering these questions, and to see what new research directions it unlocks!
We’re introducing ToyBlocks, an open simulated tabletop for controlled experiments in robot learning.
It enables robotics researchers to experimentally control training data & models, run targeted evals, & trace failures to their causes. 🧵
github.com/allenai/ToyBlocks
Ai2 retweeted
ToyBlocks is out! Huge congrats to the Ai2 Robotics team! Having watched this effort take shape at Ai2, I’ve seen firsthand how ambitious and challenging it has been—and the insight, hard work, perseverance, resilience, and resources it has taken to bring it to life. Even as an outside observer, I’ve learned so much from the team. I have enormous respect for what you’ve accomplished and am truly happy for you all. I hope this work proves valuable to the entire robotics community!
We’re introducing ToyBlocks, an open simulated tabletop for controlled experiments in robot learning.
It enables robotics researchers to experimentally control training data & models, run targeted evals, & trace failures to their causes. 🧵
github.com/allenai/ToyBlocks
Ai2 retweeted
Sharing what I spent the last year working on before starting my PhD. Glad it’s finally released! 🧱🤖
Robot foundation models have made huge progress with scale, but it’s getting harder to tell which choices actually drive the gains. I strongly believe many design choices are still understudied: data mixtures, input modalities, architectures, training recipes, and now how agentic frontier models should plan and act through robot skills.
Answering that takes systematic ablations, which need a system where you can 1. design tasks 2. generate data in a controllable way, and 3. train and evaluate models end to end.
That’s what ToyBlocks is built for. Excited to see what the community learns with it!
We’re introducing ToyBlocks, an open simulated tabletop for controlled experiments in robot learning.
It enables robotics researchers to experimentally control training data & models, run targeted evals, & trace failures to their causes. 🧵
github.com/allenai/ToyBlocks
Ai2 retweeted
This was one of the last projects I contributed to in my PhD and it’s awesome to see it finally released!
With scale we need better tools to make more fine-grained analysis of our models and algorithms. Complex-looking tasks may be cool and high-value, but are not always the highest signal for research and iteration.
Simulation of an “easy to simulate” but “difficult to master” domain of tasks lets us get more scale with minimal sim-to-real gaps, while letting you test on a incredibly diverse set of problems. From non-prehensile movements / dexterity, to higher-level reasoning.
Stay tuned to what AI2 is working on to advance open-source robotics!
We’re introducing ToyBlocks, an open simulated tabletop for controlled experiments in robot learning.
It enables robotics researchers to experimentally control training data & models, run targeted evals, & trace failures to their causes. 🧵
github.com/allenai/ToyBlocks