Pinned Tweet
๐ค What if robots could reason not just in words, but through the dynamics of the physical world?
๐ Introducing ๐ง๐ต๐ถ๐ป๐ธ๐ช๐ผ๐ฟ๐น๐ฑ from @NVIDIAAI !
๐ก By reasoning through shared physical dynamics, robots could learn from human experience and transfer motion intelligence across embodiments.
๐๐ผ๐ ๐ถ๐ ๐๐ผ๐ฟ๐ธ๐
โ ๐๐ต๐ฎ๐ถ๐ป-๐ผ๐ณ-๐๐๐ป๐ฎ๐บ๐ถ๐ฐ๐ (๐๐ผ๐): A shared motion reasoning space across human and robot embodiments.
โ ๐ช๐ผ๐ฟ๐น๐ฑ ๐๐บ๐ฎ๐ด๐ถ๐ป๐ฎ๐๐ถ๐ผ๐ป: Imagines future world representations conditioned on CoD.
โ ๐ฅ๐ ๐ณ๐ฟ๐ผ๐บ ๐ช๐ผ๐ฟ๐น๐ฑ ๐๐บ๐ฎ๐ด๐ถ๐ป๐ฎ๐๐ถ๐ผ๐ป: Reinforces reasoning through alignment between imagined and observed futures, enabling learning from human and robot videos.
๐ช๐ต๐ฎ๐ ๐ถ๐ ๐ฒ๐ป๐ฎ๐ฏ๐น๐ฒ๐
โ Robust real-world manipulation under unseen objects and environments.
โ Human-guided skill composition for novel multi-step robot tasks.
โ Robust generalization under task and environment shifts in simulation.
๐ ๐ข๐๐ฟ ๐๐ถ๐๐ถ๐ผ๐ป: Scale generalizable motion intelligence through human videos toward a unified System-1 reasoning layer for Physical AI.
Paper, robot demos & more below! ๐
Fu-En (Fred) Yang retweeted
Excited to share RoboSkills ๐ค! A minimal, low-cost skillset for Real2Sim, Agent-as-Policy, Agentic Simulation, and much more.
Its lightweight design enables lower overhead, faster iteration, and greater flexibility in agentic robotics development. ๐
We introduce RoboSkills: a minimal, low-cost skillset for agentic robotics ๐ค
It supports diverse robotics workflows, such as learning catโs cradle ๐งถ from a human video (Real2Sim) and transferring multi-person motion to characters and humanoids ๐ (Motion Generation).
We also support Auto Research, Agent as Policy, and Agentic Simulation. With suitable tools and workflows, we enable cheaper models to solve complex tasks.
GitHub: github.com/openrobokit/robosโฆ
Website: openrobokit.com/roboskills
Watch the video for more examples.
Fu-En (Fred) Yang retweeted
The best model we tested on MolmoSpace-Commonsense succeeds just 9.3% of the time.
Robotics needs benchmarks that go beyond task success to test commonsense reasoning at every step just like how we humans do things.
Introducing MolmoSpace-Commonsense, led by @arjungru! ๐ค
Weโre nowhere near saturation. More results for GPT-6, Opus, and other frontier models are coming soon!
Fu-En (Fred) Yang retweeted
๐ค What if robots could reason not just in words, but through the dynamics of the physical world?
๐ Introducing ๐ง๐ต๐ถ๐ป๐ธ๐ช๐ผ๐ฟ๐น๐ฑ from @NVIDIAAI !
๐ก By reasoning through shared physical dynamics, robots could learn from human experience and transfer motion intelligence across embodiments.
๐๐ผ๐ ๐ถ๐ ๐๐ผ๐ฟ๐ธ๐
โ ๐๐ต๐ฎ๐ถ๐ป-๐ผ๐ณ-๐๐๐ป๐ฎ๐บ๐ถ๐ฐ๐ (๐๐ผ๐): A shared motion reasoning space across human and robot embodiments.
โ ๐ช๐ผ๐ฟ๐น๐ฑ ๐๐บ๐ฎ๐ด๐ถ๐ป๐ฎ๐๐ถ๐ผ๐ป: Imagines future world representations conditioned on CoD.
โ ๐ฅ๐ ๐ณ๐ฟ๐ผ๐บ ๐ช๐ผ๐ฟ๐น๐ฑ ๐๐บ๐ฎ๐ด๐ถ๐ป๐ฎ๐๐ถ๐ผ๐ป: Reinforces reasoning through alignment between imagined and observed futures, enabling learning from human and robot videos.
๐ช๐ต๐ฎ๐ ๐ถ๐ ๐ฒ๐ป๐ฎ๐ฏ๐น๐ฒ๐
โ Robust real-world manipulation under unseen objects and environments.
โ Human-guided skill composition for novel multi-step robot tasks.
โ Robust generalization under task and environment shifts in simulation.
๐ ๐ข๐๐ฟ ๐๐ถ๐๐ถ๐ผ๐ป: Scale generalizable motion intelligence through human videos toward a unified System-1 reasoning layer for Physical AI.
Paper, robot demos & more below! ๐
Fu-En (Fred) Yang retweeted
Robots can now perform an increasingly wide range of manipulation tasks. But how far are we from robot assistants that can operate autonomously in the open world?
We introduce RobotAgentBench (RAB), a comprehensive benchmark for evaluating open-world robot autonomy in realistic household environments.
We evaluate 8 VLA models, 3 WAMs, and diverse robot agent harness designs across five dimensions: task discovery, context reasoning, manipulation, failure recovery, and robustness. Four open-ended household scenarios test how these capabilities work together to fulfill high-level user needs.
What we find:
- Manipulation โ autonomy. Standalone VLA/WAM policies struggle with task discovery and open-ended scenarios. Agent harnesses substantially improve performance, but reliable open-world autonomy remains challenging.
- Hybrid Code-VLA/WAM Policy performs best. Combining learned policies with code-based control achieves the strongest performance among evaluated configurations.
- RSI further improves robot agents. Refining reusable tools and accumulating experience lead to additional gains across iterations.
- Best couple under our default agent setting: Antigravity CLI (Gemini 4 Argon) + InternVLA-A1.5. This pairing performs best among the planner-executor combinations we evaluated, achieving an open-ended score of 10.4, compared with 8.4 for fully code-based Codex (GPT-6 Astra) and 1.1 for standalone ฯ0.5.
Thanks @_akhaliq for sharing our work!
Paper: arxiv.org/abs/2610.37666
Webpage: thinkworld-project.github.ioโฆ
ThinkWorld
World Imagination Reinforces Chain-of-Dynamics Reasoning
paper: huggingface.co/papers/2610.3โฆ
Fu-En (Fred) Yang retweeted
ThinkWorld
World Imagination Reinforces Chain-of-Dynamics Reasoning
paper: huggingface.co/papers/2610.3โฆ
Fu-En (Fred) Yang retweeted
ThinkWorld is a vision-language-action model that reasons about robot motion in a space shared by human and robot bodies, then imagines future world states to check that reasoning.
Per the creators, reinforcement learning aligns the imagined futures with what videos actually show, so it can learn from both human and robot videos, and they report robust manipulation with unseen objects and environments.
Why it matters: learning from human videos could be a scalable way to teach robots new skills.
Paper: arxiv.org/abs/2610.37666
Credit: @FuEnYang1
For a deeper dive, check out Chaehyunโs posts below:
nitter.gq/ChaehyunK87776/status/โฆ
Introducing ThinkWorld ๐
Chain-of-Dynamics (CoD) reasoning gives a shared motion language. We reinforce this reasoning by comparing resulting imagined futures with observed outcomes.
๐ thinkworld-project.github.ioโฆ
๐งต The ideas, training, and results, step by step โ
Huge thanks to our amazing team for making ThinkWorld possible! ๐
Chaehyun Kim (@ChaehyunK87776 ), Chi-Pin Huang (@chipinhxyz ), Jisoo Kim, Min-Hung Chen (@CMHungSteven ), Yu-Chiang Frank Wang, Seungryong Kim, Fu-En Yang (@FuEnYang1 ) Grateful for this collaboration between @NVIDIAAI and @KAIST_AI ! ๐
Also accepted to NeurIPS 2026 RoboWM & PTA workshops. See you in Sydney! ๐ฆ๐บ
@nvidia
@NVIDIARobotics
Fu-En (Fred) Yang retweeted
Introducing ThinkWorld ๐
Chain-of-Dynamics (CoD) reasoning gives a shared motion language. We reinforce this reasoning by comparing resulting imagined futures with observed outcomes.
๐ thinkworld-project.github.ioโฆ
๐งต The ideas, training, and results, step by step โ
Fu-En (Fred) Yang retweeted
Internship Opportunity ๐ฃ
Weโre hiring Research Scientist Interns at NVIDIAโs newly established NVIDIA Research Lab in Singapore, Agentic Robot Intelligence Laboratory (ARIEL)!
Our focus is agentic robotics: building the next generation of agentic frameworks for robots.
nvidia.wd5.myworkdayjobs.comโฆ
Fu-En (Fred) Yang retweeted
After working on Dream-1 for the last 4 months, here's my take:
LLMs vs WAMs isn't the right framing. A model like Dream-1 will soon be a tool call for LLMs, and that gets more feasible as LLMs inevitably get faster and cheaper.
Two things we didn't put in the blog post๐งต๐
Fu-En (Fred) Yang retweeted
Thanks @_akhaliq for sharing our work!
๐ Try our interactive demo with your own LLM!
๐ฌ Demo: huggingface.co/spaces/uclanlโฆ
๐ Website: openvlharness.github.io
๐ Paper: arxiv.org/abs/2610.26860
๐ป Code: github.com/OpenVLHarness/Opeโฆ
OpenVLHarness - a Hugging Face Space by uclanlp
Visual tool-use agent on your own model
huggingface.coOpenVLHarness
Real World Visual Agentic Tool-Use with Multimodal Memory
paper: huggingface.co/papers/2610.2โฆ
Fu-En (Fred) Yang retweeted
๐ Introducing OpenVLHarness:
Turn any LLM into an evidence-based reasoner that thinks and codes in visual space.
๐ GPT-6 Astra gains +9.9 points on average over 22 datasets.
โก Beats Codex and other VL harnesses by 7.7โ8.5 points, at โค37% of their cost.
๐งต More details โ
Fu-En (Fred) Yang retweeted
NVIDIA Cosmos world models have come to life with real-time interactivity! ๐๐ค๐
We are releasing our post-training framework for real-time interactive world simulation for Physical AI. The framework builds upon Cosmos 3 through causal training, distillation, and optimized streaming inference. The post-trained models stream visual observations via autoregressive generation in response to incoming actions and scene controls.
We are also releasing example model checkpoints for robotics, autonomous vehicles, and sim-to-real transfer. The release includes optimized streaming inference code to power the models.
Cosmos world foundation models provide broad visual knowledge. Each simulation setting adds its own controls, data, and evaluation requirements. Our shared recipe and example models give researchers and developers a practical starting point for building specialized interactive world simulators.
Huge shoutout to the team for the dedication, creativity, and hard work poured into this release! I am incredibly proud of this work and feel fortunate to lead such a talented team. A special thank-you to the broader Cosmos group for building the world foundation models together. Team green ๐
I am excited to see how you build your own interactive world simulator with Cosmos!
๐ Project website: research.nvidia.com/labs/cosโฆ
๐ค Model checkpoints: huggingface.co/collections/nโฆ
๐ป Code: github.com/NVIDIA/cosmos
Fu-En (Fred) Yang retweeted
Recently, thereโs been some "P(doom)" in the robot learning community around the idea that LLMs will solve robotics. I believe LLM agents are great for planning, but handling dynamics and dexterous manipulation still requires a fast System 1โor System 0.
Inspired by @typesafeai JEVโs release of a fast System 1 model, we hope this result offers another perspective on why System 1 still mattersโand why thereโs still so much worth pursuing in robotics research.
We partnered with @robocurve @chooi_jeq , an independent evaluation company, to benchmark Dream-1. It achieved the highest score among the models evaluated on RoboDojo-RC Tier 1: 59, while being faster and cheaper. โก
Regardless of VLA, WAM, LLM, etc, they can all be complementary. Ultimately, Iโm glad to see the community keeping an open mind and exploring different ways to solve robotics. Ultimately, the objective matters more than any particular approach: making robots more accessible.
Dream-1 by @dreamlabs_ai is the first model to complete "Stand Up Bottles", a difficult robotics task that frontier LLMs struggle with.
Fu-En (Fred) Yang retweeted
Introducing Flora, an open model suite for scaling robotics data engines.
It watches robot and human video and annotate what happened (timestamped instructions) and how well it went (rewards and failure reasons).
๐ย research.nvidia.com/labs/geaโฆ
We use it to train NVIDIA GR00T. ๐งต
