Michael Xu

Five iterations and six hours. Enough to fine-tune a humanoid motion generator with RL. 📌 For anyone working on humanoid locomotion or generative robot policies, this paper is worth saving. The PPO baseline used more than 160x the data and 30x the wall-clock time, and still ended up about 13 points lower in success rate. The work comes from Zachary Olkin, William Compton and Aaron Ames at Caltech's AMBER Lab. Their Unitree G1 plans 1.24 seconds of whole-body motion directly from depth images and replans every 0.24 seconds, while a separate tracker runs at 50 Hz. The generator produces 2,728 values for every plan, which makes ordinary action-space exploration extremely inefficient. Instead of adding Gaussian noise across the whole output, they perturb the conditioning, resample the initial flow-matching noise and start the robot from states the original policy would rarely visit. The nominal conditioning is kept as the label. A critic is trained on the generator's frozen prefix encoder and assigns advantages to the collected trajectories. Those advantages weight the flow-matching loss, giving better trajectories more influence on the next version of the generator. The reward is built around terrain consistency. Penetration and contact are checked first using signed distance fields, while velocity tracking and task success only count when the planned motion is consistent with the terrain. After five iterations, success on unseen geometries and skill combinations improved by as much as 25 percentage points. On one staircase, the share of runs that selected every required skill correctly went from 0% to about 80%. The ablations are useful here. Plain behavior cloning performs poorly, removing the conditioning perturbations costs around 4 to 6 points of success, and PPO on the full 2,728-dimensional output needs far more data while still performing worse. The camera setup also shows up clearly in the numbers. With both depth cameras, box success in simulation is 100%. With only the lower camera, it falls to 57.3%, because the upper camera gives the robot enough warning to slow down before a box or staircase and speed up again afterward. The full system runs without odometry or height maps. Depth images go into the motion generator, the tracker follows the generated motion, and the fine-tuning loop improves the generator from collected rollouts. Website: zolkin1.github.io/generate-t… Paper: zolkin1.github.io/generate-t… Video: youtube.com/watch?v=U81SjJIK… —— Weekly robotics and AI insights. Subscribe free: 22astronauts.com