PHD at Shanghai Institude of Technology (SII) & HIT Research Intern at Roboparty Lab AI & Robotics
China
Joined February 2023
- Tweets51
- Following318
- Followers126
- Likes116
Pinned Tweet
Can humanoid motion generation and control be unified in a single policy?
PredActor uses a single end-to-end diffusion policy rather than separate motion generation and tracking. With proprioceptive feedback, it predicts internal future states and directly outputs actions at 50 Hz onboard a Unitree G1. Text-conditioned motion, joystick steering, disturbance response, and semantic interpolation are supported in the single policy.
Project Page: masteryip.github.io/predacto…
I’m curious how Isaac Lab handles rendering. It doesn’t look like a traditional ray-tracing renderer, since I don’t see any noise in the figure. Maybe it’s using a game-engine renderer with some trade-offs? (path tracing on)
But yes, I’ll buy it.
Lei Ye (Raymon Yip) retweeted
Three years ago, training a walking policy for this 700-muscle humanoid took 2 days, using a CPU simulator and SAC. That was for a single motion. Today, with SuperMimic and MJWarp, it learns to track 1.5 hours of motion capture in 4 hours on one 5090.
muscle-supermimic.beanpow.to…
Awesome results in contact handling!
Introducing Humanoid Mobility via Attention-based Contacts!
One policy enables diverse motions over challenging terrains without reference motions at deployment.
No manually designed transition motions
No skill switching mechanism
No manual data augmentation
13.2 minutes of data
Click a goal and go!
paper: tbd
website: leggedrobotics.github.io/ACP…
Lei Ye (Raymon Yip) retweeted
Introducing Humanoid Mobility via Attention-based Contacts!
One policy enables diverse motions over challenging terrains without reference motions at deployment.
No manually designed transition motions
No skill switching mechanism
No manual data augmentation
13.2 minutes of data
Click a goal and go!
paper: tbd
website: leggedrobotics.github.io/ACP…
Lei Ye (Raymon Yip) retweeted
With @JayHe748646 @KehanWen170077 @leggedrobotics , we push humanoid mobility to next frontier.
Achieved by a generalist contact planner + a generalist tracker + a generalized neural mapping, warmstarted from few human data.
Website: leggedrobotics.github.io/ACP…
Promo Video 👇
Lei Ye (Raymon Yip) retweeted
We’re excited to announce the official release of NVIDIA Isaac Lab 3.0. 🎉
Isaac Lab 3.0 brings a new modular and lightweight architecture along with major new capabilities across the robot learning workflow: multi-physics and multi-rendering support, training at scale, and streamlined deployment on real hardware.
With Newton integration, the new OVRTX renderer, RL post-training, and LEAPP deployment, there’s a lot to explore.
Here’s a look at what’s new 🧵
(1/7)
Lei Ye (Raymon Yip) retweeted
How can we represent whole-body interaction for humanoid robots—compactly enough for low-frequency planning, yet richly enough to preserve high-frequency feedback control?
Introducing FIBER! 🤖
FIBER learns a unified latent interface between high-level planning and whole-body control, enabling a humanoid to acquire diverse interaction behaviors from heterogeneous experts through a pretrained, frozen Behavior Foundation Model (BFM).
With a single planner and BFM, we demonstrate heavy-door pushing, box carrying, object grasping, controllable navigation, and more!
🌍 Website & Live Demo: fiberaa.github.io
📄 Paper: arxiv.org/abs/2610.24675
A thread on the ideas behind FIBER 🧵👇
Lei Ye (Raymon Yip) retweeted
The @Tesla Roadster stunts, simulated in MuJoCo! Thrusters are site actuators, the ramp suction uses adhesion actuators, and the builtin box QP decides how hard to fire each jet.
The intent representation and S0's responsiveness are crucial for a general humanoid control interface.
In PredActor, unified diffusion for motion and action generation proves responsive, reminding me that the MPC-like paradigm remains worthwhile in the Astra era.
My blog on System 0 in the Astra era:
As spatial grounding (S1/2) improves, we need dynamics-grounded controllers (S0) and expressive interfaces that let Astra steer physical interaction.
Hope this resonates or sparks disagreement. I’d love your thoughts!
haoyangweng.pages.dev/blog/2…
Like the C550 metioned in the wandering earth II: Self-organizing, self-adaptive, self-sensing, and self-programming.
cool work.
Robotics has spent years scaling demonstration data and model size. What happens when we scale EXPLORATION for robot agents?
We're incredibly excited to share RAX, an agentic robotics system that learns by scaling online exploration.
A coding agent connects directly to the robot’s perception and control APIs, learns from feedback, and builds programs that perceive, plan, and act. The model stays frozen. As exploration scales, the agent improves its programs and builds up reusable skills.
The results:
• ZERO teleoperation or action demonstrations on real robots—from insertion with 0.1 mm clearance and air-fryer manipulation to deformable objects and long-horizon mobile whole-body control.
• +37 percentage points on average over leading learned policies across five simulation benchmarks: RoboDojo, BEHAVIOR-1K, HumanoidArena, DexJoCo’s bimanual tasks, and LIBERO-Pro.
• One framework spanning single-arm, bimanual, dexterous, humanoid, and mobile robots, with new embodiments connected through APIs.
The craziest part: on each real-world embodiment, going from scratch to mastering the tasks took roughly two hours. No teleop. No model finetuning.
We’re open-sourcing the full framework AND every experiment session. We can’t wait to see what people build with it!
Project + sessions: robot-agent-exploration.com/
Paper: assets.robot-agent-explorati…
GitHub: N
Project fully led by @RunhanH and @xczhangxx who are absolutely awesome.
How it works, experiments, and robot videos below 🧵
(1/N)
Lei Ye (Raymon Yip) retweeted
Robotics has spent years scaling demonstration data and model size. What happens when we scale EXPLORATION for robot agents?
We're incredibly excited to share RAX, an agentic robotics system that learns by scaling online exploration.
A coding agent connects directly to the robot’s perception and control APIs, learns from feedback, and builds programs that perceive, plan, and act. The model stays frozen. As exploration scales, the agent improves its programs and builds up reusable skills.
The results:
• ZERO teleoperation or action demonstrations on real robots—from insertion with 0.1 mm clearance and air-fryer manipulation to deformable objects and long-horizon mobile whole-body control.
• +37 percentage points on average over leading learned policies across five simulation benchmarks: RoboDojo, BEHAVIOR-1K, HumanoidArena, DexJoCo’s bimanual tasks, and LIBERO-Pro.
• One framework spanning single-arm, bimanual, dexterous, humanoid, and mobile robots, with new embodiments connected through APIs.
The craziest part: on each real-world embodiment, going from scratch to mastering the tasks took roughly two hours. No teleop. No model finetuning.
We’re open-sourcing the full framework AND every experiment session. We can’t wait to see what people build with it!
Project + sessions: robot-agent-exploration.com/
Paper: assets.robot-agent-explorati…
GitHub: N
Project fully led by @RunhanH and @xczhangxx who are absolutely awesome.
How it works, experiments, and robot videos below 🧵
(1/N)
Lei Ye (Raymon Yip) retweeted
Can humanoids adapt to new terrain using only flat-ground references? 🤖
Meet P-BFM! 🥳
We learn a terrain-agnostic intent space with unsupervised RL and forward-backward representations, paired with a terrain-aware policy that adapts across diverse terrains—without terrain-specific demonstrations.
📄🎬💻 Check out our paper, video, and code here!
🌐 roboparty.github.io/P-BFM/
Lei Ye (Raymon Yip) retweeted
Excited to share 𝗕𝗥𝗔𝗖𝗘: a whole-body tracker for humanoids that can exert forces from diverse poses and adapt to varied terrains.
Humanoid whole-body trackers are typically trained to follow references recorded on flat ground, carrying nothing. As a result, they know 𝗻𝗲𝗶𝘁𝗵𝗲𝗿 𝘁𝗵𝗲 𝗳𝗼𝗿𝗰𝗲𝘀 𝘁𝗵𝗲 𝗿𝗼𝗯𝗼𝘁 𝗺𝘂𝘀𝘁 𝗲𝘅𝗰𝗵𝗮𝗻𝗴𝗲 𝘄𝗶𝘁𝗵 𝗼𝗯𝗷𝗲𝗰𝘁𝘀 𝗻𝗼𝗿 𝗰𝗮𝗻 𝘁𝗵𝗲𝘆 𝗮𝗱𝗮𝗽𝘁 𝘁𝗼 𝗱𝗶𝘃𝗲𝗿𝘀𝗲 𝘁𝗲𝗿𝗿𝗮𝗶𝗻𝘀. BRACE closes both gaps in a single tracker.
Our key innovation is a 𝗿𝗲𝗳𝗲𝗿𝗲𝗻𝗰𝗲-𝗮𝗱𝗮𝗽𝘁𝗮𝘁𝗶𝗼𝗻 𝗳𝗼𝗿𝗺𝘂𝗹𝗮𝘁𝗶𝗼𝗻. Given humanoid reference trajectories, we adapt them to the terrain and the commanded force, then train the controller on these adapted references. Tracking them naturally produces the stance and bracing needed to handle the terrain and exert the force (or compensate for external loads). 𝗧𝗵𝗲 𝗿𝗲𝘀𝘂𝗹𝘁𝗶𝗻𝗴 𝗰𝗼𝗻𝘁𝗿𝗼𝗹𝗹𝗲𝗿 𝗱𝗲𝗽𝗹𝗼𝘆𝘀 𝗱𝗶𝗿𝗲𝗰𝘁𝗹𝘆 𝗼𝗻 𝘁𝗵𝗲 𝗿𝗲𝗮𝗹 𝗿𝗼𝗯𝗼𝘁.
To know more, please visit our:
Project page: multiplylabs.github.io/brace…
Interactive demo: multiplylabs.github.io/robog…
More details in this thread🧵
1/n
This is what i really want to do before. the way they collect data is eye-catching.
super.
Perceiving the environment, understanding spatial constraints, and coordinating whole-body motion are essential for humanoid robots to integrate into human society—and remain major challenges in robotics.
To address this, the Galbot team introduces PASSAGE, a framework that combines environmental perception, whole-body motion planning, and closed-loop control, enabling humanoid robots to autonomously plan and execute flexible traversal motions based on their surroundings.
Built on 100 hours of human data across 1,500 scenes, it achieves 98.67% success on the Clutter Scene Traversal benchmark.
#PASSAGE #Galbot #embodiedAI #humanoidrobot