
Robot learning @LeRobotHF @huggingface. Building Physical AI @codecopenflow
Paris, France
Joined September 2020
- Tweets1.1K
- Following2.3K
- Followers668
- Likes12.2K
Cool robot demo. More interesting: adding the next RL algorithm to LeRobot is now one file + one config.
SAC was first. Who's next?
The best is yet to come. 🤖
🤖Adding new RL algorithms to LeRobot just got much easier.
Demo: HIL-SERL training with a SAC-based RL algorithm on an SO-100 for a hole-in-hand peg-in-hole task.
Sparse reward, only 30 offline demos mixed with live robot experience, and ~1 hour of online training with human interventions only when the policy fails.
The bottom graph tracks intervention rate: high at the start, steadily dropping as the policy improves.
The refactor separates algorithm logic from training infrastructure:
• RLAlgorithm owns learning logic
• RLTrainer handles orchestration
• DataMixer combines rollouts, demos, interventions, and future data sources
Adding an RL algorithm now looks much closer to adding a policy: one algorithm file, one config, one registry entry.
SAC is first. RLT, RECAP, ConRFT, QC-FQL, DSRL, and VLA RL fine-tuning next!
@Thom_Wolf @ClementDelangue
lil’km retweeted
More arms. More humanoids. 🦾
New hardware in LeRobot 0.7:
- YAM from @i2rt_robotics : drive one arm or two, with gravity feed-forward computed from the arm's own model
- reBot B601 from @seeedstudio: now on RobStride or Damiao motors, and a bimanual setup can mix the two
- @UnitreeRobotics G1 + NVIDIA SONIC: your policy outputs a 64-D motion token, and the SONIC decoder turns it into 50 Hz targets for all 29 joints. Works with remote inference
- The 23-DoF G1 joins too: same driver, one setting
Yes, that's our G1 dodging a basketball 🏀
Docs in the comments.
lil’km retweeted
We’re not going to win by lobbying, plotting, or manipulating. We’re going to win by building, in the open, with the whole community!
Today, with @ggerganov we’re super excited to announce the release of llama.cpp v1, the first major fully stable version of the most used library to run AI models locally, meaning on your phone, your laptop, and your workstation.
It means you can own your intelligence for free, not send your data to anyone and be 100% in control, wherever you are in the world. It shows our commitment to support every hardware, platform and model type out there and makes AI distributed rather than concentrated in a few Silicon Valley orgs.
Huge thanks to the team, the long-time maintainers and every contributor who helped make this happen. Let’s build! 💛🧡💚
llama.cpp v1.0.0 is now available
huggingface.co/blog/ggml-org…
lil’km retweeted
Run big policies on a remote GPU ⚡🤖
Your robot keeps moving while the GPU thinks.
New in LeRobot 0.7: the robot computer runs cameras, motors and recording, and a policy server on a GPU machine predicts the next chunk of actions. An action buffer asks for the next chunk early, so the robot doesn't stop between chunks.
Two commands:
lerobot-policy-server --config_path=server.yaml
lerobot-rollout --inference.type=remote ...
Same machine, LAN or routed networks over Zenoh, with TLS and access control. Interactive sessions, episode recording and RTC come along, and it's tested on real robots with ACT, SmolVLA, XVLA, EVO1 and LaWAM.
Docs in the comments 👇
lil’km retweeted
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter.
AI will transform every industry, power every company, and be built by every country.
Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty.
The world needs both frontier closed models and frontier open models.
images.nvidia.com/pdf/Open-W…
lil’km retweeted
Jensen Huang resigned.
Hopefully, he will be able to have it as a vacation. Recharge, recover and come back.
Open source has many strong fighters and will prevail.
lil’km retweeted
Five new VLAs in LeRobot 0.7. The zoo keeps growing 🤖
- G0.5 from @GalaxeaDynamics: a 2B VLA that writes its subtask, then its actions, in one stream. Works with /subtask and /autosteer
- LingBot-VLA 2.0 from @robbyant_brain: one model across embodiments (55-D action space, ~60,000 h of pretraining). Its sparse MoE runs an action chunk in 120 ms in LeRobot, down from 756 ms
- DM0.5 from @Dexmal_AI: a Gemma3-4B VLM + a 680M flow-matching action expert, on standard LeRobot datasets
- Wall-OSS 0.5 from @XSquareRobot: a 4B VLA that also writes text. It answers /vqa and plans its own subtasks
- FineART-VLA: π0.5 with a language head, trained on FineART (more in our language post)
Docs in the comments 👇
lil’km retweeted
10,000 hours of open egocentric data with SOTA 3D head and hand tracking from @GroundedSI are now available 🤗
RoboCap-10k spans 21 industries in 5 continents with 619 real, paid work tasks all using 6-camera/dual IMU RoboCap hardware.
Explore the dataset and research ↓
Introducing RoboCap-10k. 10,000 hours of open egocentric data with metric hand / head pose on every frame.
Captured on RoboCap across 11 countries and 21 industries.
GroundedAPI produces 3D enrichment giving robot policies high-fidelity action grounding.
with @BitRobotNetwork @huggingface @rerundotio ↓
lil’km retweeted
Praxis-1: an open-weight world action model for real robots 🤖
New from @runwayml and now in LeRobot.
Praxis-1 learns how objects behave and how tasks unfold from large-scale video, which gives it a head start on a new robot. One checkpoint adapts with light fine-tuning or LoRA: fine-tuned on Google's RT-1 robot, which it never saw in training, it reaches 80.0% on SimplerEnv. Weights, inference and fine-tuning code are open under Apache 2.0, commercial use included, and it runs on NVIDIA from H100 to Jetson Thor.
Model, blog and docs in the comments 👇
Today, we’re making Praxis-1 available to everyone. Our state-of-the-art world action model turns video pretraining into control for real robots, and it ships with open weights under Apache 2.0, ready to run and fine-tune on your own hardware.
lil’km retweeted
Robots that imagine before they act 🔮🤖
World action models predict what happens next, then act. LeRobot 0.7 adds four of them:
- Cosmos3 from @NVIDIARobotics: 16.5B Nano and 4.6B Edge, 97.9% LIBERO average. Edge runs on one GPU
- FLUX 3 Action from @bfl_ai: one open 7B diffusion transformer predicts future video and actions together, with an SO-101 checkpoint
- LaWAM from RLinf: imagines the future in latent space (DINOv3 features), 98.5% on LIBERO measured in LeRobot
- Long-WAM from NVIDIA: looks back over 19.2 s of video history, 99.0% on LIBERO measured in LeRobot
Docs in the comments 👇
lil’km retweeted
We just crossed:
- 20M AI builders on @huggingface
- 300PB of models, datasets, agent traces, and RL environments hosted on the @huggingface hub (9PB added by users just last week)
Also October was our biggest month ever in terms of DAUs, MAUs and revenue.
Let's go open-source AI!
lil’km retweeted
Hugging Face is the best place to host and share 3D assets - drop in .glb, .ply, .stl, or splat files and get a 3D gallery automatically.
Congrats @nvidiaomniverse on achieving 1M downloads of SimReady 3D assets!
To publish your own 3D datasets with an interactive gallery, check out the docs: huggingface.co/docs/hub/data…
1 million downloads of SimReady assets on @HuggingFace. 🎉
SimReady assets help teams use 3D content across supported tools and simulation workflows, from robot workcells to AI factory digital twins.
Thank you to everyone experimenting, sharing feedback and building with SimReady assets.
Find assets for your next simulation in SimReady Central: huggingface.co/spaces/nvidia…
lil’km retweeted
It took me ~21 hours to build @asimovinc from @menloresearch 🤖
Ever wondered what it takes to go from a pile of parts to a walking humanoid? Watch it !
Full timelapse in the reply 👇
Built at @huggingface with heart 🤗
lil’km retweeted
Train on terabytes. Don't download them. 📦
LeRobot 0.7 trains straight from the Hub:
- Streaming: each worker fetches only the video bytes it needs. 0.52% of step time spent waiting for data (π0.5, 4× H100, 10.7 TB dataset)
- LanceDB, built with @lancedb: frames are read in place. 1.7% waiting for data, vs 37.4% from local disk after a 384 GB download
- GPU augmentations: 2.8× faster ACT training on an H100, with 8 new robotics augmentations
Plus a training stack for big models: FSDP2 + HSDP, checkpoints saved 3.81× faster and resumable on any GPU count, FP16, EMA and Trackio.
--dataset.streaming=true
Docs in the comments 👇
lil’km retweeted
Talk to your robot while it works 🗣️🤖
New in LeRobot 0.7: interactive rollouts. Start a policy, then steer it mid-run, without a reset:
- /subtask: switch what it's doing
- /vqa: ask what it sees
- /autosteer: give it a goal and let a planner pick the steps
- record episodes while you steer
lerobot-rollout --interactive=true
No text-generating policy? Bring any VLM as the planner: transformers serve, vLLM or Inference Providers.
And FineART-VLA, trained on FineART from @scale_AI and Hugging Face (1,718 h, 533,913 subtask labels), says its next subtask out loud, then does it.
Docs in the comments 👇
lil’km retweeted
Infinite Factory 🏭
An endless robot factory, generated live in your browser with Wave Function Collapse.
Every robot, conveyor and box is a SimReady OpenUSD asset streamed from the HF Hub, rendered with huggingface/usd-web-viewer
Try it: hf.co/spaces/mishig/simready…
lil’km retweeted
LeRobot v0.7.0 is out, and this is a very big one 🦾
You can now talk/steer a robot while it’s running; change its task:
`/subtask pick up the red cube`
Ask what it sees:
`/vqa how many cubes are left?`
Or give it a higher-level goal:
`/autosteer clear the table`
and let the policy or an external VLM planner figure out the subtasks!
And this just scratches the surface of v0.7.0! Other banger features include 👇
Training directly on huge datasets from the Hub, Storage Buckets or @lancedb without downloading them first; particularly relevant when the Hub is already approaching !!300 TB of open Physical AI data!!
The training stack has also been rebuilt for 5-20B parameter world action models: FSDP2, HSDP, distributed checkpoints, FP16, EMA and more.
The model zoo keeps growing:
- 4 new world action models, including Cosmos 3 and FLUX 3 Action
- 4 new VLAs
- remote inference, so the robot can stay lightweight while the policy runs on a GPU server
- a common simulation stack that works much more like real hardware
LeRobot is becoming less about individual robot policies and more about having one open stack from TBs of demonstrations, all the way to a robot you can interact with in real time!




















